Comments on: SQL SERVER – Select and Delete Duplicate Records – SQL in Sixty Seconds #036 – Video

By: Anuj Grewal

Anuj Grewal — Fri, 12 Apr 2013 10:41:48 +0000

DELETE
FROM TestTable
WHERE ID NOT IN
(
SELECT MAX(ID)
FROM TestTable
GROUP BY NameCol)
this is not working ……………

By: Mandar Shindagi

Mandar Shindagi — Thu, 07 Mar 2013 12:31:55 +0000

Sanjay, I think you are overcomplicating this. If you have a single column table, do a SELECT DISTINCT into a temp table, truncate your single column table and INSERT the distinct values back from temp table.
You may want to go one step further by adding a PK constraint for future proofing.
Hope this helps.

Pinal, nice post mate.

By: DoronF

DoronF — Wed, 06 Mar 2013 22:36:23 +0000

Hi Pinal

I appreciate the info in this article.

Removing duplicate records could be a tedious job especially when you have hundreds of millions of records in one table. When removing duplicate records you may want to combine different fields’ values in one good record. Say you have a Phone Number in one records and DOB in another record but they are duplicate based on First, Last and Address.

Here is an article that will discuss the above issues and how to resolve the problem while integrating all required fields in one good record. http://www.dfarber.com/computer-consulting-blog/2011/12/26/remove-duplicate-records-in-sql.aspx
Any feedback is appreciated.

Regards,

Doron

By: SQL SERVER – Fix: Error: 1505 The CREATE UNIQUE INDEX statement terminated because a duplicate key was found for the object name and the index name « SQL Server Journey with SQL Authority

Tue, 05 Feb 2013 01:30:44 +0000

[...] On Separate note, here is the blog post with video which explains how you can delete the duplicate row from the table: Select and Delete Duplicate Records – SQL in Sixty Seconds #036 – Video. [...]

By: Sanjay Monpara

Sanjay Monpara — Fri, 21 Dec 2012 05:34:50 +0000

Very good Scott
Its perfect!!!

By: Scott McPhaed

Scott McPhaed — Thu, 20 Dec 2012 19:13:29 +0000

You can also use newID():

WITH cte(NameCol, RankField)
AS (SELECT NameCol
, RankField = DENSE_RANK()
OVER (
PARTITION BY NameCol
ORDER BY newID())
FROM
TestTable)
DELETE FROM cte
WHERE RankField > 1

By: Sanjay Monpara

Sanjay Monpara — Thu, 20 Dec 2012 05:27:44 +0000

One doubt,
we have to pass where condition to update records, then here possibility to update same row_num for duplicate records, isn’t it?

By: Ali Kolahdoozan

Ali Kolahdoozan — Thu, 20 Dec 2012 04:20:03 +0000

How could I user Row_Number in Delete Statement ?.If we did not have ID column , we are not able to delete duplicated record.

By: Garrett

Garrett — Thu, 20 Dec 2012 03:03:44 +0000

You don’t need to append a new column in the table, that is incorrect. You can utilize the row_number() and delete the duplicate records without creating a new column on the table.

By: Kaz

Kaz — Wed, 19 Dec 2012 15:57:21 +0000

You will need to append a new column in your table, and then update it with row_number over (partition by namecol order by namecol), and then execute a delete command, delete from tabel where row_num > 1

By: Sanjay Monpara

Sanjay Monpara — Wed, 19 Dec 2012 12:52:00 +0000

suppose single column table cantains records like below

NameCol
——–
First
Second
Second
Second
Second
Third

Then how its possible to delete duplicate records using row_number() function.
even if SSMS not allows to delete single record from duplicate set & showing error.

By: Naren

Naren — Wed, 19 Dec 2012 10:44:10 +0000

@Sanjay

You can use ROW_NUMBER() function in this case

By: Sanjay Monpara

Sanjay Monpara — Wed, 19 Dec 2012 06:46:20 +0000

In oracle we can use ROWID Pseudocolumn,
It helps to delete records even if table not contains incremental unique id & without using another temp table.

Is there any method in sql server to delete records in such situation without using another(temp) table?