Why it applies order by before the top?:)

the increment(Identity) is incremented first
or the constraints are checked first
As I know the answer that increments comes first but I need a reason for it..
because if the increment is done first and then constraints are checked if constraints are not matched then still the identity is increased which is wrong according to me..
but why these happens please reply as soon as possible

One of the best!

He also discussed that when TOP is there ORDER BY is executed along with it. If it does not exist, ORDER BY is executed separate.

More detail I will post soon.

Great session.

Kind Regards,
Pinal

“just having a clustered key present doesn’t mean that the TOP results will use that key.”

That’s something the help file states.

But if the CLUSTERED INDEX is used, that is “probably” the order. According to Denny Cherry “If you don’t have an ORDER BY, then the SQL Server will probably sort by the clustered index and use it for filtering. If there is no clustered index it would then fall back to using the table itself and return the data in the order it comes across it.” From: http://itknowledgeexchange.techtarget.com/sql-server/back-to-basics-whats-the-difference-between-a-scan-and-a-seek/

So, technically it doesn’t have to use it. And sure, you probably shouldn’t rely on it. But the fact remains, it is used. And blaming bugs in the code on “techncially”s is silly. Unless it really is the case, but my gut feeling is otherwise.

I just reread the challenge. To have the query use the clustered index and not return cluster key ordered rows?

No, I don’t think I’ll attempt that.

But the query above shows that just having a clustered key present doesn’t mean that the TOP results will use that key. In the case above the Non-clustered index is used even w/o an ORDER BY.

>> Can you give an example …uses the CLUSTERED INDEX but the ordering is otherwise?

Something like this:
CREATE TABLE A (Col varchar(900) NOT NULL, Data varchar(4) NULL
, CONSTRAINT [PK_A] PRIMARY KEY CLUSTERED ([Col] ASC)
)
GO
INSERT INTO A (Col, Data) VALUES (’2kakfvbsaib83y34nlnlkakfvbsaib83y34nlnlkakfvbsaib83y34nlnlkakfvbsaib83y34nlnlkakfvbsaib83y34nlnlkakfvbsaib83y34nlnlkakfvbsaib83y34nlnlkakfvbsaib83y34nlnlkakfvbsaib83y34nlnlkakfvbsaib83y34nlnlkakfvbsaib83y34nlnlkakfvbsaib83y34nlnlkakfvbsaib83y34nlnlkakfvbsaib83y34nlnlkakfvbsaib83y34nlnlkakfvbsaib83y34nlnlkakfvbsaib83y34nlnlkakfvbsaib83y34nlnlkakfvbsaib83y34nlnlkakfvbsaib83y34nlnlkakfvbsaib83y34nlnlkakfvbsaib83y34nlnlkakfvbsaib83y34nlnlkakfvbsaib83y34nlnlkakfvbsaib83y34nlnlkakfvbsaib83y34nlnlkakfvbsaib83y34nlnlkakfvbsaib83y34nlnlkakfvbsaib83y34nlnlkakfvbsaib83y34nlnlkakfvbsaib83y34nlnlkakfvbsaib83y34nlnlkakfvbsaib83y34nlnlkakfvbsaib83y34nlnlkakfvbsaib83y34nlnlkakfvbsaib83y34nlnlkakfvbsaib83y34nlnlkakfvbsaib83y34nlnlkakfvbsaib83y34nlnlkakfvbsaib83y34nlnlkakfvbsaib83y34nlnlkakfvbsaib83y34nlnlkakfvbsaib83y34nlnlkakfvbsaib83y34nlnlkakfvbsaib83y34nlnlkakfvbsaib83y34nlnl1akfvbsaib83y34nlnmhsojhg’, ‘B’)
INSERT INTO A (Col, Data) VALUES (’1kakfvbsaib83y34nlnlkakfvbsaib83y34nlnlkakfvbsaib83y34nlnlkakfvbsaib83y34nlnlkakfvbsaib83y34nlnlkakfvbsaib83y34nlnlkakfvbsaib83y34nlnlkakfvbsaib83y34nlnlkakfvbsaib83y34nlnlkakfvbsaib83y34nlnlkakfvbsaib83y34nlnlkakfvbsaib83y34nlnlkakfvbsaib83y34nlnlkakfvbsaib83y34nlnlkakfvbsaib83y34nlnlkakfvbsaib83y34nlnlkakfvbsaib83y34nlnlkakfvbsaib83y34nlnlkakfvbsaib83y34nlnlkakfvbsaib83y34nlnlkakfvbsaib83y34nlnlkakfvbsaib83y34nlnlkakfvbsaib83y34nlnlkakfvbsaib83y34nlnlkakfvbsaib83y34nlnlkakfvbsaib83y34nlnlkakfvbsaib83y34nlnlkakfvbsaib83y34nlnlkakfvbsaib83y34nlnlkakfvbsaib83y34nlnlkakfvbsaib83y34nlnlkakfvbsaib83y34nlnlkakfvbsaib83y34nlnlkakfvbsaib83y34nlnlkakfvbsaib83y34nlnlkakfvbsaib83y34nlnlkakfvbsaib83y34nlnlkakfvbsaib83y34nlnlkakfvbsaib83y34nlnlkakfvbsaib83y34nlnlkakfvbsaib83y34nlnlkakfvbsaib83y34nlnlkakfvbsaib83y34nlnlkakfvbsaib83y34nlnlkakfvbsaib83y34nlnlkakfvbsaib83y34nlnlkakfvbsaib83y34nlnmhsojhg’, ‘A’)
GO
CREATE NONCLUSTERED INDEX [NCIX_Data_DESC] ON [dbo].[A] ([Data] DESC)
GO
SELECT TOP 2 * FROM A

The point is that when the BOL says that TOP returns arbitrary rows in the absence of ORDER BY, they are describing the expected behavior and not trying to give us insight into the implementation. The implementation, confronted with a clustered index without an ORDER BY, may typically return an ordered set; but we are not supposed to perceive it as an ordered set because there may be at least one codepath through the optimizer that doesn’t order as we would expect.

pinaldave seems to have a knack for finding those code paths :)

I can also give an example where step #2 ON is evaluated after step #8 SELECT or STEP #4 WHERE.

Don’t rely on implementation whenever and wherever you can get away with it.

Can you give an example where there is a CLUSTERED INDEX, no ORDER BY, and the query uses the CLUSTERED INDEX but the ordering is otherwise?

Kind Regards,
Pinal

In the case of ORDER BY and TOP, Brian Tkatch gave you the only answer that you can hang your hat on; that is, that in the absence of an ORDER BY clause, TOP will return arbitrary rows. Therefore, you can consider that ORDER BY precedes TOP when ORDER BY is specified, however, that may not be what actually happens procedurally, just what apparently happens when the results come back. I can conceive of times where an optimizer could know enough about the data to select the TOP rows before even making a JOIN. Like gravity, Microsoft’s implementation is a black box to us: we all can make use of gravity everyday without having to know how it works. In this case, we are better off not knowing how it works, because we don’t want to be implementation-specific when we write our SQL. I can produce sample code that shows these differences if you’re interested.

P.S. It would be easy for someone to read Brian Tkatch’s comment and think that an ORDER BY is not required in the case where you pull results from a single table with a clustered index and the cluster key order is the order you desire. The fact is that result ordering is never guaranteed in the absence of an ORDER BY clause. On a related note, rows are not physically order by cluster key. The clustered index is a structure that is built from the data pages.

but turns out to be very dangerous
if used incorrectly that is the query may take hours to execute and the final result contains lakhs of unwanted rows.

Simple way to implement join
on tables all having one field in common
say ID,i.e. ID field links all tables to each other

Sample simple query could be used for n number of tables and n number of joins

select

a.ID,
a.field2,
a.field3,
b.field1,
b.field2,
c.field1,
c.field2,
d.field1,
d.field2,
d.field3,
d.field4

from

(

(select * from table1) as a

left join

(select * from table2) as b

on a.ID=b.ID

left join

(select * from table3) as c

on a.ID=c.ID

left join

(select * from table4) as d

on a.ID=d.ID

)

I suggest never use left join !

If have to , create index on field like ID in above sample
on all tables to reduce query time

It would appear as though to do the top it has to know the Order By clause and it needs to understand there is an Order By, and then filter only the top number/percent of rows according to that.

The actual ordering for display or passing on to the next query happens after the TOP filter is applied (even though it looked at the order by).

One important thing to remember is that we are talking about the logical query processing here. The physical order of events is not necessarily (not, actually) the same always.

Any more ideas?

Regards,
Pinal

http://www.sql.co.il/books/insidetsql2008/Logical%20Query%20Processing%20Poster.pdf

Those examples definitely help show that order by happens first. The query plan will definitely show it as well.

As for a simple way to think about it, just think about what happens with each operator.

The ORDER BY is ordering your results, it is forcing them to come back in a certain way (contrary to relational theory :) ). The TOP operator is telling SQL Server to restrict the rows returned to a certain number of rows (either explicitly or by percentage). If TOP happened first that would mean you would just be ordering by the number of rows specified in the top. If you were to query a table and ask for the TOP 1 ORDERED BY a certain value that would be silly for a couple reasons:

1.) Silly to order by one row
2.) Let’s say you were querying a column that was not the clustered key, the TOP isn’t going to guarantee you which row you get (as described above) so not doing the ORDER BY first means you are not going to be getting meaningful results.

Good post and always helpful to remind folks the order of logical query processing (physical processing doesn’t always follow that order but it helps people think of ways to better limit their results earlier and improve performance.

Itzik’s book INSIDE SQL Server 2005 T-SQL Querying describes each of these steps in detail (though I believe that is actually Lubar’s chapter if memory serves correct)

You are answering like Imran, who used to answer really nice.

Very nice!

Regards,
Pinal

Example:

First for an example, we want to get the query to read the TABLE in a different order than the ORDER BY. To do that, the data should be stored opposite the ORDER BY. That can be accomplished via a CLUSTERED INDEX:

CREATE TABLE #Data
(
Id_Clustered TINYINT,
Id_Ordered TINYINT,
CONSTRAINT Data_Id_PK PRIMARY KEY CLUSTERED (Id_Clustered)
);
INSERT INTO #Data(Id_Clustered, Id_Ordered)
SELECT 1, 2 UNION ALL
SELECT 2, 1;

Id_Clustered is the COLUMN the TABLE will be CLUSTERED on, whereas Id_Ordered will be in the ORDER BY of the query. The values in the two records are opposites. This will force a query without an ORDER BY and with an ORDER BY to be different:

SELECT
Id_Clustered,
Id_Ordered
FROM
#Data;
Yeilds:
1,2
2,1

Because that is how it is CLUSTERED. But, if we add an ORDER BY:

SELECT
Id_Clustered,
Id_Ordered
FROM
#Data
ORDER BY
Id_Ordered;

The result is reversed to:
2,1
1,2

because it is following the ORDER BY statement.

Add a TOP 1 for the test to see which one gets pulled:

SELECT TOP 1
Id_Clustered,
Id_Ordered
FROM
#Data
ORDER BY
Id_Ordered;

Which shows:

2,1

This is the same as the ORDER BY results.

The Query plan shows the Clustered Index Scan first and then a Sort (Top N Sort). It specifically reordered the results to follow the ORDER BY.

This example shows that ORDER BY is handled first, and then TOP.