Comments on: SQL SERVER – Delete Duplicate Records – Rows

By: Aditya

Aditya — Tue, 10 Nov 2009 15:39:19 +0000

Pinal rather than adding a ID col and dropping it I think WHILE loop (which you already suggested) will be better opeion and even better than that is use ROW_NUMBER() function and simply delete the records which has row_number count > 1

select row_number() over (partition by col order by col from tbl) and just delete where row_number() is greater than one and if one can set rowcount 1 also so if 2 records are exactly identical only 1 will be deleted….

thanks

By: Amit Dhall

Amit Dhall — Tue, 10 Nov 2009 08:30:05 +0000

delete from tablename where NOT IN ( Select distinct * from tablename)

By: Joe

Joe — Thu, 05 Nov 2009 06:43:17 +0000

Hi Sir
i use Microsoft access 2003
how to make the same record not to calculate the total or can i count the same record and devided by the count so that i can get original sum account
because duplicate record sum it all up
which i only want to have only one record

By: Great Man

Great Man — Thu, 01 Oct 2009 10:58:55 +0000

Very nice and I am proud of you. I am very excited.

By: Avineet

Avineet — Thu, 27 Aug 2009 01:02:53 +0000

HI Manoj, you can do so by setting row count 1 and put the delete statement, this way you will delete only 1 record of the 10 duplicate records. Avineet

By: Manoj

Manoj — Mon, 24 Aug 2009 17:37:45 +0000

I want to delete the duplicate record ( i.e i have 10 duplicate record , i want to delete a particular record leaving all the remaining 9 records means , how can i doit , plz reply me )

By: ryan

ryan — Mon, 17 Aug 2009 14:04:44 +0000

Pinal,

Thanks a ton, I spent a day experimenting with how to do this before I came across your solution, I added a key to my table and this worked like a charm.

By: Rich

Rich — Mon, 10 Aug 2009 16:12:25 +0000

You could use CTE to delete the duplicates if you have SQL 2005 or above.

Create the duplicate table: ———————————–
IF OBJECT_ID(‘SalesHistory’) IS NOT NULLDROP TABLE SalesHistoryCREATE TABLE [dbo].[SalesHistory] ( [Product] [varchar](10) NULL, [SaleDate] [datetime] NULL, [SalePrice] [money] NULL ) GOINSERT INTO SalesHistory(Product, SaleDate, SalePrice)SELECT ‘Computer’,'1919-03-18 00:00:00.000′,1008.00UNION ALLSELECT ‘BigScreen’,'1927-03-18 00:00:00.000′,91.00UNION ALLSELECT ‘PoolTable’,'1927-04-01 00:00:00.000′,139.00UNION ALLSELECT ‘Computer’,'1919-03-18 00:00:00.000′,1008.00UNION ALLSELECT ‘BigScreen’,'1927-03-25 00:00:00.000′,92.00UNION ALLSELECT ‘PoolTable’,'1927-03-25 00:00:00.000′,108.00UNION ALLSELECT ‘Computer’,'1919-04-01 00:00:00.000′,150.00UNION ALLSELECT ‘BigScreen’,'1927-04-01 00:00:00.000′, 123.00UNION ALLSELECT ‘PoolTable’,'1927-04-01 00:00:00.000′, 139.00UNION ALLSELECT ‘Computer’,'1919-04-08 00:00:00.000′, 168.00
=======================================

Remove the duplicates —————————————

;WITH SalesCTE(Product, SaleDate, SalePrice, Ranking)AS(SELECT Product, SaleDate, SalePrice,Ranking = DENSE_RANK() OVER(PARTITION BY Product, SaleDate, SalePrice ORDER BY NEWID() ASC)FROM SalesHistory)DELETE FROM SalesCTEWHERE Ranking > 1

By: Zoltan

Zoltan — Mon, 10 Aug 2009 13:09:13 +0000

Brian,

Thank you for your help! Worked like a charm!

Thanks!

By: Brian Tkatch

Brian Tkatch — Fri, 07 Aug 2009 17:29:17 +0000

@Zoltan

Remove “calownerid” from the GROUP BY clause.

By: Zoltan

Zoltan — Fri, 07 Aug 2009 14:24:34 +0000

Hello Pinal,

Could you please help me? ..When I ran my script it removed both of the records. Could you please advice me why?
I do not have index in that table.

Thank you!

drop table dupcalref;
create temporary table dupcalref
SELECT MAX(calownerid) as id FROM calref
group by calownerid, calenderid having count(calownerid)>1;

alter table dupcalref add index(id);

delete FROM calref where calownerid in
(select id from dupcalref);

By: siddique ahmad

siddique ahmad — Tue, 04 Aug 2009 11:52:42 +0000

Thanks a lot this really help me in removing the duplicate invoice added due to my code

By: ARNAB SARKAR

ARNAB SARKAR — Mon, 03 Aug 2009 05:59:41 +0000

TABLE- MASTER,W_CODE IS COLUMN NAME.

DELETE FROM MASTER WHERE W_ID IN(SELECT MAX(W_ID) FROM MASTER WHERE W_CODE IN(SELECT W_CODE FROM MASTER GROUP BY W_CODE HAVING COUNT(W_CODE)>1))

By: Neto

Neto — Wed, 29 Jul 2009 04:54:59 +0000

Thanks for your help man!!!

It helps me a lot!!!

Greats from Lima, Perú.

By: aasim abdullah

aasim abdullah — Mon, 13 Jul 2009 06:26:21 +0000

excellent post, Senthilnathan comments really helped me to delete duplicate rows with most appropriate way.

By: Lucky Ahuja

Lucky Ahuja — Fri, 03 Jul 2009 09:56:29 +0000

Hey Guys thats all fine . U can store the values in the Temp table remove the duplicate records and empty the original table . Then move all the unique records to the original table . But its a lengthy process suppose i am having 2 million records in a table and from that table if i want to ren=move the duplicate records then it would really affect the performance . Sorry Since i am not having so much of exp and if i had hurted someone . Pinal Sir do let me know if i am wrong

By: amit

amit — Tue, 23 Jun 2009 09:09:01 +0000

Hello sir,

You r genious, I love your articles.

thanks

By: amit jain

amit jain — Tue, 16 Jun 2009 10:39:09 +0000

Hello Pinal,

Is there any procedure to restore deleted records from particular table in database?

Thank you
Amit Jain

By: gpshiburaj

gpshiburaj — Tue, 09 Jun 2009 09:55:34 +0000

Thanks Brian,
I was thinking that we would have to use a cursor for this, using row_number is easier.

By: Brian Tkatch

Brian Tkatch — Mon, 08 Jun 2009 16:15:46 +0000

@gpshiburaj

ROW_NUMBER() does that:

WITH
Data(field01)
AS
(
SELECT ‘AA’ UNION ALL
SELECT ‘AB’ UNION ALL
SELECT ‘AC’ UNION ALL
SELECT ‘AB’ UNION ALL
SELECT ‘AA’ UNION ALL
SELECT ‘AD’
)
SELECT
field01,
ROW_NUMBER() OVER(PARTITION BY field01 ORDER BY field01)
FROM
Data;