a) Use a batch approach like the methods described in this article to remove duplicates
b) Use a real-time approach by adding a unique index to prevent duplicates when you insert?

Thanks,
David

duplicate rows=case 1-case2

SELECT Personentityid,COUNT(NAMESNO) AS DUPLICATE INTO DBA_DB.dbo.duplicatevalues FROM DBA_DB.dbo.SpellingVariation
GROUP BY Personentityid

case 1:
select sum(duplicate) from DBA_DB.dbo.duplicatevalues where duplicate>1

Case 2:
select Personentityid from DBA_DB.dbo.duplicatevalues where duplicate>1

What do you mean by “top 3″
SELECT…FROM TABLE
EXCEPT
SELECT TOP(3) …FROM TABLE ORDER BY…

any sqlquery please help me…madan

The tables I am looking at text import tables. They are all very wide with upwards of 50 columns, all of the columns are varchar(255).

( I didn’t create the table but I have to maintain it! )

Duplicates would only exists if all 50 of the columns were identical.

The current query is a SELECT DISTINCT, but I wonder whether or not a CTE would be more efficient.

Maybe if I added a computed column that’s a hash of all the columns, and looked for duplicates based on that column?

Can you just check this and make me sure it is correct?

GO
SET ANSI_PADDING OFF
———————————————————————————

Data Insertion section:

INSERT INTO dbo.tBLtEST VALUES
(’1′,’R2′,’R3′,’R4′,’R5′,NULL,NULL,NULL,NULL,NULL)–NORMAL RECORD
INSERT INTO dbo.tBLtEST VALUES
(‘A’,'RR’,'RR3′,’RG4′,’RG5′,’RH6′,’RH7′,’RTJ’,NULL,NULL)–NORMAL RECORD
INSERT INTO dbo.tBLtEST VALUES
(‘B’,'RB2′,’RB3′,’RB4′,’RB5′,NULL,’KJL’,'LKJU’,'BBNJ’,'TEST’)–NORMAL RECORD
INSERT INTO dbo.tBLtEST VALUES
(‘C’,'R2′,’R3′,’R4′,’R5′,’R6′,’R7′,NULL,NULL,NULL)–DUPLCATE RECORD WITH THE FIRST RECORD
INSERT INTO dbo.tBLtEST VALUES
(‘D’,'DR2′,’DR3′,’DR4′,’DR5′,NULL,NULL,NULL,NULL,NULL)
INSERT INTO dbo.tBLtEST VALUES
(‘E’,'DR2′,’DR3′,’DR4′,’DR5′,’KK’,'NN’,NULL,NULL,NULL)–DUPLICATE WITH ABOVE RECORD (pk=D)
INSERT INTO dbo.tBLtEST VALUES
(‘F’,'RF2′,’RF3′,’FR4′,’FR5′,’FFFF’,'OOO’,NULL,’MMM’,'KKL’)–NORMAL RECORD
INSERT INTO dbo.tBLtEST VALUES
(‘G’,'RGG2′,’GGGR3′,’RGGG4′,’GGGR5′,NULL,’GGG’,'GHH’,NULL,NULL)–NORMAL RECORD

———————————————————-

what i need is to remove the duplicates, but always remove the ones which the
R6 and R7 field are filled.
here in this sample i want to be sure that the Records C and E are removed.
hope it is clear now and you can guid me for that.
PS: please consider that I have Huge amount of data in the DB.
BRs,

The problem is: the system by mistake inserted duplicate data in R2-R5 (R1 is PK assuem autonumber) and in one of these duplicates R6 and R7 are also filled.
I want to remove the duplicates, which their R6,R7 are filled.
Consider that the R6,R7,… may be filled for other data, and they are ok!
by this method i could find them and remove the duplicates, but i cannot decide to remove which one! I mean I can not force to remove the duplicates which has R6,R7.
Please advice.

http://msdn.microsoft.com/en-us/library/ms190766.aspx

delete
from dbo.tablename
where tablename.%%physloc%%
NOT IN (Select min(b.%%physloc%%)
from dbo.tablename b
group by b.columnwithdups);

Could you give me help on the following?
1. How many parameters that CTE can accepts?
2. Shall we able to move duplicate data into another table and How to delete duplicate records simultaneously using CTE.? Let me explain my actual scenario. I have a table called Test.

Test has the following data
Col1 Col2 Col3
—— —— ——
101 abc xyz
102 aaa bbb
101 abc xyz
101 abc xyz
103 xxx yyy
101 abc xyz
101 abc xyz
101 abc xyz
101 abc xyz
101 abc xyz
:::::::::::::::::::::::::

I need to store the duplicate records in the table Test_duplicate. Could you tell me how to achieve it with CTE.

My require output is

Test:

Col1 Col2 Col3
—— —– —–
101 abc xyz
102 aaa bbb
103 xxx yyy

Test_duplicate:
Col1 Col2 Col3
—– —– —–
101 abc xyz
101 abc xyz
101 abc xyz
101 abc xyz
101 abc xyz
101 abc xyz
101 abc xyz
::::::::::::::::::::::::

Thanks in advance.

Thanks,
U.Penchal Reddy.