My company uses GoldMine CRM (on SQL Server), and I'm writing a query to find duplicate records. The query is great, but it returns two rows for each duplicate, and I only need one. However, I can't seem to use Row_Number() at all - it always returns a blank column. Here's my query:
SELECT *
FROM (
SELECT
c11.company AS Company1,
c12.company AS Company2,
c11.phone1 AS DuplicatePhone,
c11.address1 AS C1Address1,
c12.address1 AS C2Address1,
c11.zip AS Zip1,
c12.zip AS Zip2,
c11.contact AS Contact_1,
c12.contact AS Contact_2,
SUBSTRING(c11.company, 1, CHARINDEX(' ', c11.company)) AS C1_Firstword,
SUBSTRING(c12.company, 1, CHARINDEX(' ', c12.company)) AS C2_Firstword,
c11.accountno AS Acctno1,
c12.accountno AS Acctno2
FROM db.contact1 AS c11
INNER JOIN db.contact1 AS c12
ON c11.phone1 = c12.phone1
WHERE c11.state = 'MA'
AND c12.state = 'MA'
AND c11.phone1 IS NOT NULL
AND c11.phone1 <> ''
AND c11.accountno <> c12.accountno) AS foo
WHERE (PATINDEX('%' + foo.C1_Firstword + '%', foo.company2) > 0
OR PATINDEX('%' + foo.C2_Firstword + '%', foo.company1) > 0)
ORDER BY foo.DuplicatePhone
The query first looks for records with the same phone number, and then looks for similarities in the company name (sometimes our contacts share a phone number without being duplicates, but it's common to find duplicates where one name is 'John Smith Enterprises' and the other 'Smith Enterprises')
I've tried every iteration of ROW_NUMBER() in this query and in a far simpler one, eg:
SELECT c1.accountno, c1.company, ROW_NUMBER() OVER(ORDER BY c1.Accountno ASC) row_num
FROM bpmain1.dbo.contact1 c1
WHERE c1.state = 'MA'
... and I always get a blank column. My theory is that the SQL panel in GoldMine is stopping me from using it, since the results that I get back from GoldMine always include a column 'Row' that's numbered (As though GoldMine "conveniently" wraps every query with an empty ROW_NUMBER() clause.)
So, I end up with two rows for each duplicate instance, and I only need one - it doesn't matter which one. The point of using ROW_NUMBER() was to get me every other result. Any other ideas?
Just change your where conditions:
FROM db.contact1 c11 INNER JOIN
db.contact1 c12
ON c11.phone1 = c12.phone1 AND c11.state = c12.state
WHERE c11.state = 'MA' AND
c11.phone1 <> '' AND
c11.accountno < c12.accountno
--------------------^ this is the key change
That is, you don't need to remove duplicates. Just don't generate them in the first place by returning the contacts in account order.
Note that the condition c11.phone1 IS NOT NULL is redundant. Both the JOIN conditions and the <> '' filter out NULL values.
In Oracle 12 (and APEX) I am having problems with a CASE statement in a WHERE clause. The scenario is a master table, ORDER, and a PRODUCTS_BOUGHT table, so this is a one to many relationship. I have a report, with a filter on PRODUCTS_BOUGHT. The filter populates a bind variable/APEX page item called :P36_PRODUCT_LISTING. If the user selects a given product, I want the report to just show those orders which contain the given product. The filter contains the word 'All', which should not do any filtering, as well as each product we carry.
My SQL statement is
Select distinct
:P36_PRODUCT_LISTING,
LISTAGG(product_name, ', ') WITHIN GROUP (ORDER BY product_name) OVER (PARTITION BY B.SALES_ORDER_NUMBER) AS products,
...
from ORDER A, PRODUCTS_BOUGHT B
where
A.SALES_ORDER_NUMBER = B.SALES_ORDER_NUMBER
and
case when
:P36_PRODUCT_LISTING = 'All' then 1 = 1
else a.SALES_ORDER_NUMBER IN (SELECT SALES_ORDER_NUMBER from PRODUCTS_BOUGHT where PRODUCT_NAME = :P36_PRODUCT_LISTING) end
When I run the statement, the error I get is Missing Keyword.
What am I doing wrong?
Don't use case. Just use boolean logic:
where (:P36_PRODUCT_LISTING = 'All' or
a.SALES_ORDER_NUMBER IN (SELECT SALES_ORDER_NUMBER
from PRODUCTS_BOUGHT
where PRODUCT_NAME = :P36_PRODUCT_LISTING
)
)
The problem with the case (as you have written it) is that Oracle does not treat the value from a logical expression as a valid value. Some databases do, but not Oracle.
In addition:
Don't use commas in the from clause. Always use proper, explicit join syntax.
Use table aliases that are abbreviations for the table names. Much easier to read the queries and fix bugs.
select distinct should not be necessary here. You are doing an aggregation without a group by, so there is only one row anyway.
So:
Select :P36_PRODUCT_LISTING,
LISTAGG(b.product_name, ', ') WITHIN GROUP (ORDER BY b.product_name) OVER (PARTITION BY B.SALES_ORDER_NUMBER) AS products,
...
from ORDER o join
PRODUCTS_BOUGHT B
on p.SALES_ORDER_NUMBER = B.SALES_ORDER_NUMBER
where (:P36_PRODUCT_LISTING = 'All' or
o.SALES_ORDER_NUMBER IN (SELECT SALES_ORDER_NUMBER
from PRODUCTS_BOUGHT
where PRODUCT_NAME = :P36_PRODUCT_LISTING
)
);
Case is designed to return a value, not a statement..
Try OR instead
where :P36_PRODUCT_LISTING = 'All'
or (:P36_PRODUCT_LISTING <> 'All'
and a.SALES_ORDER_NUMBER IN (SELECT SALES_ORDER_NUMBER from PRODUCTS_BOUGHT where PRODUCT_NAME = :P36_PRODUCT_LISTING))
I want to get a parameter. The priority for getting that parameter is that I have to look for it in Table1, but if it doesn't exist there, I have to look for it in Table2. If not, so that parameter is null (this situation should not happen, but, well, there is always an edge case).
I wanted to try something like this:
SELECT NVL(
SELECT paramValue from Table1
where paramName = "paramName" and Id = "id",
SELECT paramValue from Table2
where paramName = "paramName" and Id = "id")
But it gives me a syntax error.
Is there any way of doing something like this?
Enclose the sub-queries in their own set of parentheses, like this:
SELECT NVL((SELECT Atomic_Number FROM Elements WHERE Name = 'Tungsten'),
(SELECT Atomic_Number FROM Elements WHERE Name = 'Helium'))
FROM sysmaster:informix.sysdual;
74
SELECT NVL((SELECT Atomic_Number FROM Elements WHERE Name = 'Wunderkind'),
(SELECT Atomic_Number FROM Elements WHERE Name = 'Helium'))
FROM sysmaster:informix.sysdual;
2
SELECT NVL((SELECT Atomic_Number FROM Elements WHERE Name = 'Wunderkind'),
(SELECT Atomic_Number FROM Elements WHERE Name = 'Helios'))
FROM sysmaster:informix.sysdual;
The last query generated a NULL (empty line) as output, which is mimicked by a non-breaking space on the last line.
Granted, I'm not selecting from two tables; that's immaterial to the syntax, and the sub-queries would work on two separate tables as well as on one table.
Tested with Informix 12.10.FC6 and CSDK 4.10.FC6 on Mac OS X 10.11.5.
There's another way:
SELECT * FROM (
SELECT paramValue from Table1
where paramName = "paramName" and Id = "id"
union all
SELECT paramValue from Table2
where paramName = "paramName" and Id = "id"
) x
LIMIT 1
Which is IMHO easier to read.
I created this CTE that returns first and last names from 2 different tables. I would like to use the CTE to identify all of the records that have the same last names and the first name of one column starts with the same first letter of another column.
This is an example of the results of the CTE. I want the SELECT using the CTE to return only the highlighted results:
;WITH CTE AS
(
SELECT AD.FirstName AS AD_FirstName, AD.LastName AS AD_LastName, NotInAD.FirstName As NotInAD_FirstName, NotInAD.LastName As NotInAD_LastName
FROM PagingToolActiveDirectoryUsers AD JOIN
(
SELECT FirstName, LastName
FROM #PagingUsersParseName
EXCEPT
SELECT D.FirstName, D.LastName
FROM PagingToolActiveDirectoryUsers D
WHERE D.FirstName <> D.LastName AND D.LastName <> D.LoginName
AND D.LoginName LIKE '%[0-9]%[0-9]%'
) AS NotInAD ON NotInAD.LastName = AD.LastName
)
SELECT *
FROM CTE
WHERE (AD_LastName = NotInAD_LastName) AND (AD_FirstName LIKE ('''' + SUBSTRING(NotInAD_FirstName, 1, 1) + '%'''))
ORDER BY AD_LastName, AD_FirstName;
The result of this query returns no rows.
What am I doing wrong?
Thanks.
You're enclosing the string to be searched for with single-quotes, but it doesn't appear that the data in AD_FirstName has those single-quotes embedded in it. I suggest you replace the first line of the WHERE clause with
WHERE (AD_LastName = NotInAD_LastName) AND (AD_FirstName LIKE (SUBSTRING(NotInAD_FirstName, 1, 1) + '%'))
Best of luck.
I am creating a key-wording module where I want to search data using the comma separated words.And the search is categorized into comma , and minus -.
I know a relational database engine is designed from the principle that a cell holds a single value and obeying to this rule can help for performance.But in this case table is already running and have millions of data and can't change the table structure.
Take a look on the example what I exactly want to do is
I have a main table name tbl_main in SQL
AS_ID KWD
1 Man,Businessman,Business,Office,confidence,arms crossed
2 Man,Businessman,Business,Office,laptop,corridor,waiting
3 man,business,mobile phone,mobile,phone
4 Welcome,girl,Greeting,beautiful,bride,celebration,wedding,woman,happiness
5 beautiful,bride,wedding,woman,girl,happiness,mobile phone,talking
6 woman,girl,Digital Tablet,working,sitting,online
7 woman,girl,Digital Tablet,working,smiling,happiness,hand on chin
If search text is = Man,Businessman then result AS_ID is =1,2
If search text is = Man,-Businessman then result AS_ID is =3
If search text is = woman,girl,-Working then result AS_ID is =4,5
If search text is = woman,girl then result AS_ID is =4,5,6,7
What is the best why to do this, Help is much appreciated.Thanks in advance
I think you can easily solve this by creating a FULL TEXT INDEX on your KWD column. Then you can use the CONTAINS query to search for phrases. The FULL TEXT index takes care of the punctuation and ignores the commas automatically.
-- If search text is = Man,Businessman then the query will be
SELECT AS_ID FROM tbl_main
WHERE CONTAINS(KWD, '"Man" AND "Businessman"')
-- If search text is = Man,-Businessman then the query will be
SELECT AS_ID FROM tbl_main
WHERE CONTAINS(KWD, '"Man" AND NOT "Businessman"')
-- If search text is = woman,girl,-Working the query will be
SELECT AS_ID FROM tbl_main
WHERE CONTAINS(KWD, '"woman" AND "girl" AND NOT "working"')
To search the multiple words (like the mobile phone in your case) use the quoted phrases:
SELECT AS_ID FROM tbl_main
WHERE CONTAINS(KWD, '"woman" AND "mobile phone"')
As commented below the quoted phrases are important in all searches to avoid bad searches in the case of e.g. when a search term is "tablet working" and the KWD value is woman,girl,Digital Tablet,working,sitting,online
There is a special case for a single - search term. The NOT cannot be used as the first term in the CONTAINS. Therefore, the query like this should be used:
-- If search text is = -Working the query will be
SELECT AS_ID FROM tbl_main
WHERE NOT CONTAINS(KWD, '"working"')
Here is my attempt using Jeff Moden's DelimitedSplit8k to split the comma-separated values.
First, here is the splitter function (check the article for updates of the script):
CREATE FUNCTION [dbo].[DelimitedSplit8K](
#pString VARCHAR(8000), #pDelimiter CHAR(1)
)
RETURNS TABLE WITH SCHEMABINDING AS
RETURN
WITH E1(N) AS (
SELECT 1 UNION ALL SELECT 1 UNION ALL SELECT 1 UNION ALL SELECT 1 UNION ALL SELECT 1 UNION ALL
SELECT 1 UNION ALL SELECT 1 UNION ALL SELECT 1 UNION ALL SELECT 1 UNION ALL SELECT 1
)
,E2(N) AS (SELECT 1 FROM E1 a, E1 b)
,E4(N) AS (SELECT 1 FROM E2 a, E2 b)
,cteTally(N) AS(
SELECT TOP (ISNULL(DATALENGTH(#pString), 0)) ROW_NUMBER() OVER (ORDER BY (SELECT NULL)) FROM E4
)
,cteStart(N1) AS(
SELECT 1 UNION ALL
SELECT t.N+1 FROM cteTally t WHERE SUBSTRING(#pString, t.N, 1) = #pDelimiter
),
cteLen(N1, L1) AS(
SELECT
s.N1,
ISNULL(NULLIF(CHARINDEX(#pDelimiter, #pString, s.N1),0) - s.N1, 8000)
FROM cteStart s
)
SELECT
ItemNumber = ROW_NUMBER() OVER(ORDER BY l.N1),
Item = SUBSTRING(#pString, l.N1, l.L1)
FROM cteLen l
Here is the complete solution:
-- search parameter
DECLARE #search_text VARCHAR(8000) = 'woman,girl,-working'
-- split comma-separated search parameters
-- items starting in '-' will have a value of 1 for exclude
DECLARE #search_values TABLE(ItemNumber INT, Item VARCHAR(8000), Exclude BIT)
INSERT INTO #search_values
SELECT
ItemNumber,
CASE WHEN LTRIM(RTRIM(Item)) LIKE '-%' THEN LTRIM(RTRIM(STUFF(Item, 1, 1 ,''))) ELSE LTRIM(RTRIM(Item)) END,
CASE WHEN LTRIM(RTRIM(Item)) LIKE '-%' THEN 1 ELSE 0 END
FROM dbo.DelimitedSplit8K(#search_text, ',') s
;WITH CteSplitted AS( -- split each KWD to separate rows
SELECT *
FROM tbl_main t
CROSS APPLY(
SELECT
ItemNumber, Item = LTRIM(RTRIM(Item))
FROM dbo.DelimitedSplit8K(t.KWD, ',')
)x
)
SELECT
cs.AS_ID
FROM CteSplitted cs
INNER JOIN #search_values sv
ON sv.Item = cs.Item
GROUP BY cs.AS_ID
HAVING
-- all parameters should be included (Relational Division with no Remainder)
COUNT(DISTINCT cs.Item) = (SELECT COUNT(DISTINCT Item) FROM #search_values WHERE Exclude = 0)
-- no exclude parameters
AND SUM(CASE WHEN sv.Exclude = 1 THEN 1 ELSE 0 END) = 0
SQL Fiddle
This one uses a solution from the Relational Division with no Remainder problem discussed in this article by Dwain Camps.
From what you've described, you want the keywords that are included in the search text to be a match in the KWD column, and those that are prefixed with a - to be excluded.
Despite the data existing in this format, it still makes most sense to normalize the data, and then query based on the existence or non existence of the keywords.
To do this, in very rough terms:-
Create two additional tables - Keyword and tbl_Main_Keyword. Keyword contains a distinct list of each of the possible keywords and tbl_Main_Keyword contains a link between each record in tbl_Main to each Keyword record where there's a match. Ensure to create an index on the text field for the keyword (e.g. the Keyword.KeywordText column, or whatever you call it), as well as the KeywordID field in the tbl_Main_Keyword table. Create Foreign Keys between tables.
Write some DML (or use a separate program, such as a C# program) to iterate through each record, parsing the text, and inserting each distinct keyword encountered into the Keyword table. Create a relationship to the row for each keyword in the tbl_main record.
Now, for searching, parse out the search text into keywords, and compose a query against the tbl_Main_Keyword table containing both a WHERE KeywordID IN and WHERE KeywordID NOT IN clause, depending on whether there is a match.
Take note to consider whether the case of each keyword is important to your business case, and consider the collation (case sensitive or insensitive) accordingly.
I would prefer cha's solution, but here's another solution:
declare #QueryParts table (q varchar(1000))
insert into #QueryParts values
('woman'),
('girl'),
('-Working')
select AS_ID
from tbl_main
inner join #QueryParts on
(q not like '-%' and ',' + KWD + ',' like '%,' + q + ',%') or
(q like '-%' and ',' + KWD + ',' not like '%,' + substring(q, 2, 1000) + ',%')
group by AS_ID
having COUNT(*) = (select COUNT(*) from #QueryParts)
With such a design, you would have two tables. One that defines the IDs and a subtable that holds the set of keywords per search string.
Likewise, you would transform the search strings into two tables, one for strings that should match and one for negated strings. Assuming that you put this in a stored procedure, these tables would be table-value parameters.
Once you have this set up, the query is simple to write:
SELECT M.AS_ID
FROM tbl_main M
WHERE (SELECT COUNT(*)
FROM tbl_keywords K
WHERE K.AS_ID = M.AS_ID
AND K.KWD IN (SELECT word FROM #searchwords)) =
(SELECT COUNT(*) FROM #searchwords)
AND NOT EXISTS (SELECT *
FROM tbl_keywords K
WHERE K.AS_ID = M.AS_ID
AND K.KWD IN (SELECT word FROM #minuswords))