Insert Missing Values Into Table Using SQL - sql

Question

Objective: I need to fully populate a table with a matrix of values for each column by [PropertyId] grouping. Several [PropertyId] have all the necessary values for each column (Table 1), however, many are missing some values (Table 2). Furthermore, not every [PropertyId] needs these values as they have completely different regional values. Therefore, I need to identify which [PropertyId] both need the values populated and don't have all the necessary values.
Examples:
Table 1. Each identified [PropertyId] grouping should have 23 distinct records for these four columns [ReportingVolumeSettingId],[SpeciesGroupInventoryID],[CropCategoryID],[SortOrder].
Table 2. Here is an example of a PropertyID that is missing a value combination as it only has 22 records:
Both of these example results were queried from the same table [ReportingVolume]. I have not been successful in even identifying which record combination per [PropertyID] are missing. I would like to identify each missing record combination and then insert that record combination into the [ReportingVolume] table.
Problem to Solve -- The SQL Code below is my attempt to 1. Identify the correct List of Values; 2. Identify which properties should have matching values; 3. Identify which properties are missing values; 4. Identify the missing values per property.
;with CORRECT_LIST as
(
select
SpeciesGroupInventoryName, SpeciesGroupInventoryId, CropCategoryName,CropCategoryID, UnitOfMeasure, SortOrder
--*
from [GIS].[RST].[vPropertyDefaultTimberProductAndUnitOfMeasure]
where PropertyId in (1)
)
,
property_list as
(
select distinct rvs.propertyid as Volume_Property, pd.PropertyName, pd.PropertyId from RMS.GIS.ReportingVolumeSetting rvs
right outer JOIN RMS.GIS.PropertyDetail AS pd ON rvs.PropertyId = pd.PropertyId
left outer JOIN RMS.GIS.SpeciesGroupInventory AS sgi ON rvs.SpeciesGroupInventoryId = sgi.SpeciesGroupInventoryId
where sgi.SpeciesGroupInventoryId in (1,2,3)
or pd.PropertyId = 171
)
, Partial_LISTS as
(
select Count(distinct ReportingVolumeSettingId) as CNT_REPORT, pd.PropertyName, pd.PropertyId
from [GIS].[ReportingVolumeSetting] rvs
right outer JOIN property_list AS pd ON rvs.PropertyId = pd.PropertyId
group by pd.propertyId, pd.PropertyName
)
, Add_Props as
(
select propertyName, propertyId, SUM(CNT_REPORT) as CNT_RECORDS from Partial_LISTS
where CNT_REPORT < 23
group by propertyName, propertyId
)
, RVS_RECORDS_PROPS as
(
select addProps.PropertyName, rvs.* from [GIS].[ReportingVolumeSetting] rvs
join Add_Props addProps on addprops.PropertyId = rvs.PropertyID
where rvs.PropertyId in (select PropertyId from Add_Props)
)
select rp.PropertyName, cl.*, rp.SpeciesGroupInventoryId from correct_list cl
left outer join RVS_Records_Props rp
on rp.SpeciesGroupInventoryId = cl.SpeciesGroupInventoryId
and rp.CropCategoryId = cl.CropCategoryID
and rp.SortOrder = cl.SortOrder
Order by rp.PropertyName
How can I modify the code or create a new code block identifies the missing values and inserts them into the table per PropertyId?
I am using SQL SMSS v15.
Thanks so much.

Answer 1

This should identify missing entries. You could simply add an INSERT INTO command on top of this. Keep in mind as the ReportingVolumeSettingId is unique and unknown it's not covered here.
SELECT * FROM (SELECT DISTINCT PropertyId FROM ReportingVolume) rv
CROSS APPLY
(
SELECT DISTINCT SpeciesGroupInventoryId
, CropCategoryId
, SortOrder
FROM ReportingVolume
) x
EXCEPT
SELECT PropertyId, SpeciesGroupInventoryId, CropCategoryId, SortOrder FROM ReportingVolume

Answer 2

I don't have access to your data, so I cannot provide an example specific to your environment, but I can provide you a simple example using SQL Server's EXCEPT operator.
Run the following example in SSMS:
-- Create a list of required values.
DECLARE #Required TABLE ( required_id INT, required_val VARCHAR(50) );
INSERT INTO #Required ( required_id, required_val ) VALUES
( 1, 'Required value 1.' ), ( 2, 'Required value 2.' ), ( 3, 'Required value 3.' ), ( 4, 'Required value 4.' ), ( 5, 'Required value 5.' );
-- Create some sample data to compare against.
DECLARE #Data TABLE ( PropertyId INT, RequiredId INT );
INSERT INTO #Data ( PropertyId, RequiredId ) VALUES
( 1, 1 ), ( 1, 2 ), ( 1, 3 ), ( 2, 1 ), ( 2, 2 ), ( 2, 4 ), ( 2, 5 );
-- Set a property id value to query.
DECLARE #PropertyId INT = 1;
-- Preview #Data's rows for the specified #PropertyId.
SELECT * FROM #Data WHERE PropertyId = #PropertyId ORDER BY PropertyId, RequiredId;
At this point, I've created a list of required values (required_id 1 through 5) and some dummy data to check them against. This initial SELECT shows the current resultset for the specified #PropertyID:
+------------+------------+
| PropertyId | RequiredId |
+------------+------------+
| 1 | 1 |
| 1 | 2 |
| 1 | 3 |
+------------+------------+
You can see that required_id values 4 and 5 are missing for the current property. Next, we can compare #Required against #Data and INSERT any missing required values using the EXCEPT operator and then return the corrected resultset.
-- Insert any missing required values for #PropertyId.
INSERT INTO #Data ( PropertyId, RequiredId )
SELECT #PropertyId, required_id FROM #Required
EXCEPT
SELECT PropertyId, RequiredId FROM #Data WHERE PropertyId = #PropertyId;
-- Preview #Data's revised rows for #PropertyId.
SELECT * FROM #Data WHERE PropertyId = #PropertyId ORDER BY PropertyId, RequiredId;
The updated resultset now looks like the following:
+------------+------------+
| PropertyId | RequiredId |
+------------+------------+
| 1 | 1 |
| 1 | 2 |
| 1 | 3 |
| 1 | 4 |
| 1 | 5 |
+------------+------------+
You can run this again against #PropertyId = 2 to see a different scenario.
Note the order to using EXCEPT. The required rows comes first, followed by the EXCEPT operator, and then the current rows to be validated. This is important. EXCEPT is saying show me rows from #Required that are not in #Data--which allows for the inserting of any missing required values into #Data.
I know this example doesn't represent your existing data with the 23 rows requirement, but hopefully, it will get you moving with a solution for your needs. You can read more about the EXCEPT operator here.
Here is the complete SSMS example:
-- Create a list of required values.
DECLARE #Required TABLE ( required_id INT, required_val VARCHAR(50) );
INSERT INTO #Required ( required_id, required_val ) VALUES
( 1, 'Required value 1.' ), ( 2, 'Required value 2.' ), ( 3, 'Required value 3.' ), ( 4, 'Required value 4.' ), ( 5, 'Required value 5.' );
-- Create some sample data to compare against.
DECLARE #Data TABLE ( PropertyId INT, RequiredId INT );
INSERT INTO #Data ( PropertyId, RequiredId ) VALUES
( 1, 1 ), ( 1, 2 ), ( 1, 3 ), ( 2, 1 ), ( 2, 2 ), ( 2, 4 ), ( 2, 5 );
-- Set a property id value to query.
DECLARE #PropertyId INT = 1;
-- Preview #Data's rows for the specified #PropertyId.
SELECT * FROM #Data WHERE PropertyId = #PropertyId ORDER BY PropertyId, RequiredId;
-- Insert any missing required values for #PropertyId.
INSERT INTO #Data ( PropertyId, RequiredId )
SELECT #PropertyId, required_id FROM #Required
EXCEPT
SELECT PropertyId, RequiredId FROM #Data WHERE PropertyId = #PropertyId;
-- Preview #Data's revised rows for #PropertyId.
SELECT * FROM #Data WHERE PropertyId = #PropertyId ORDER BY PropertyId, RequiredId;