SELECT *

מכירים את זה שאתם מגיעים לחתום על חוזה התקשרות עם חברה מסוימת, התנאים נראים מבטיחים, הצעה מעולה ממש עסקת חייכם, אבל אז מגיעים לסוף הדף, ושם באותיות טל ומטר יש כוכבית (או כמה) אשר מבהירות לכם שלא מדובר בעסקת חייכם ולא נעליים. היום כל בר דעת יודע- יש להיזהר מהכוכבית!! אז מעתה והלאה, אני מציע לכם בחום ליישם את הלקח הזה גם בכל הקשור לסביבת ה- Production של SQLServer!! בסביבה זו יש להימנע ככל האפשר מהשימוש ב- *.

אחרי שבפעם האחרונה עסקתי בהרגל מגונה שעיקרו הוספת NOLOCK לכל שאילתה בלי חשבון, היום אעסוק בהרגל מגונה אחר, נפוץ הרבה יותר והוא כאמור * SELECT . בפוסט זה אתייחס לנושא ממספר זוויות והן: הסיבה העיקרית לשימוש ב- *, מתי השימוש לא מזיק, ועיקר הפוסט, ההשפעות השליליות של * SELECT , כן, יש כאלו, ולא מעט!

כאשר מנסים להצדיק את השימוש ב- * SELECT הטענה ראשונה וכמעט היחידה שעולה היא ש- * SELECT חוסך עבודה. נגיד שיש טבלה עם 5 עמודות, ומחר  צריך להוסיף עמודה נוספת, אז לא צריך לשנות את הקוד יותר מדיי, משום ש – * SELECT לוקח את כל העמודות וזה חוסך זמן, אין צורך  לתקן את הקוד ע"י הוספת העמודות. הטענה השנייה היא דומה, אם ישנן הרבה עמודות בטבלה מסוימת אז זה יותר קל ונוח לרשום * במקום להקליד את כל העמודות. הצד השווה בין שתי הטענות הוא ששורש השימוש ב- * טמון בעצלנות (או חסכון בזמן אם להיות עדינים יותר…) אז אם אלו הסיבות העיקריות לשימוש ב- *, אז למי שלא מכיר ישנה אפשרות ב- Management Studio להוסיף את כל העמודות הינף קליק. ב- Object Explorer, לכל טבלה יש יש אותן שש עמודות קבועות, אחת מהן היא Columns. עומדים על תיקייה בשם  Columns ופשוט גוררים אותה אל המיקום הרצוי בחלון ה- Query. וזהו! לא צריך להקליד כלום, כל העמודות כתבות לפניכם, מופרדות בפסיקים על מגש של Query.

אז לאחר ששללתי את השימושים העיקריים ב * SELECT, אדגיש כי במקרים לא מעטים השימוש ב * SELECT הוא בסדר גמור ולגיטימי, לדוגמא כאשר רוצים לבצע בדיקות מסוימות או שעובדים עם DMV's שונים. אין שום בעיה במקרים אלו. הבעיות מתחילות שעושים שימוש ב * SELECT בסביבת ה- Production, לדוגמא: * SELECT בפרוצדורות שונות. מה שמוביל אותי לדבר על הצד הפחות זוהר של * SELECT .

ההשפעות השליליות של * SELECT :

• פגיעה בשימוש נכון באינדקסים קימיים- במקרים מסוימים כאשר נריץ שאילתה עם * SELECT , התוצאה הישירה תהיה סקירה מלאה של טבלה או של ה- Clustered Index.  לשם המחשה נריץ את שתי השאילתות הבאות, שימו לב השאילתות זהות במהותן, רק שבראשונה יש שימוש ב- *, ואילו בשנייה כתובות העמודות הרצויות בלבד:

:Using SELECT * query –

* SELECT

FROM Sales.SalesOrderDetail

;WHERE ProductID < 750

:Same query as the above but this time, using specific columns –

SELECT SalesOrderDetailID,ProductID,SalesOrderID

FROM Sales.SalesOrderDeta

;WHERE ProductID < 750

נציץ ב- Execution plan:

לחץ להגדלה

השאילתה הראשונה (עם * ), עולה כמעט פי 50 מהשאילתה שמציינת במפורש שדות ספציפיים. הסיבה לכך נעוצה בעובדה ששאילתה מספר 1 גורמת לסקירה מלאה של ה- Clustered Index ( שווה ערך פחות או יותר סריקה מלאה של הטבלה). ואילו בשאילתה מספר 2 בוצעה סריקה של ה- Index NonClustered שבמקרה זה, היא פעולה הרבה פחות כבדה. זאת ועוד, במקרים אחרים, שימוש ב- * SELECT גורר פעולות מיותרות אחרות(וכבדות!!) כמו Lookups. לדוגמא אם נריץ את אותן שאילתות מהדוגמא הקודמת אך נשנה את התנאי ב- WHERE ל-  20 אז נוכל לראות ששאילתה מספר 1 גורמת לפעולת lookups שהיא בעצם החלק הכבד של השאילתה (50% מעלות השאילתה מוקדשת לכך). ואילו באפשרות השנייה, מתבצע Index seek על NonClustered IX_SalesOrderDetail_ProductID, שהיא יעילה בהרבה.

לחץ על התמונה להגדלה

• יצירת עומס על השרת- אז כפי שהרגע ראינו במקרים רבים שימוש ב * SELECT גורם לסריקות של טבלאות או אינדקסים בצורה לא יעילה ולא נחוצה. ישנם מקרים רבים בהם השאילתה מחזירה לאפליקציה נתונים שאין בהם שום שימוש, משום שהשאילתה מחזירה 10 עמודות לדוגמא, אך האפליקציה מציגה רק 3 עמודות מתוך ה- 10. העומס מתבטא הן בפעולות I/O רבות שאיו בהן צורך, והן במעבר של נתונים ברשת שגם בהם כמובן אין שום שימוש.
• שינוי סכמה ב- View-  כאשר יוצרים View בעזרת * SELECT ישנה בעיה עיקרית אחת. נניח שלאחר זמן מה, מחליטים לשנות את שם אחת העמודות מ- Bla ל- Bla2. ה-MetaDta של ה- View לא מתעדכן באופן אוטומטי על השינוי, ואין לו מושג שעמודה מסוימת כבר לא נקראת כפי שהיא נקראה בעבר. נסבך את העניינים עוד טיפה, תארו לכם שעכשיו נוסיף עמודה חדשה שנקראת Bla והיא בכלל מסוג אחר(עמודת תאריך במקום מספר שהיה עד כה)במצב זה אנו נקבל ונציג מידע שגוי. (אציין כי ניתן למנוע מצב זה בעזרת שימוש ב- SCHEMABINDING אשר לא מאפשר שימוש ב- * SELECT).
• · ניטור שגיאות- על מנת להמחיש את הנקודה הבאה אגש ישר לדוגמא ישנה פרוצדורה שמקבלת כפרמטר את rowguid@  ומחזירה נתונים:

* SELECT

FROM Sales.SalesOrderDetail

WHERE rowguid = @rowguid

מחר, הטבלה תעבור שינוי ועמודת RowGuid תימחק מן הטבלה. מי שמחק את הטבלה בטוח שלא צריך לעשות שום שינוי ב- SQL משום שהוא שולף נתונים בעזרת * SELECT. אבל משום שהסינון נעשה על עמודה שלא קיימת עוד, נקבל הודעת שגיאה, ואם אין ניטור מסודר של הודעות השגיאה, אז נקבל הודעת שגיאה עלומה, ועכשיו נצטרך להפוך עולמות עד אשר נגלה את הבעיה האמיתית.

לסיכום, * SELECT נוח לשימוש ויעיל כל עוד לא משתמשים בו באופן קבוע בסביבת Production. בסביבה זו, יש לשאוף למצב שלא יהיה כלל שימוש ב * SELECT. אם וכאשר נגיע למצב שבו אין שימוש ב * SELECT, אנו נרוויח מכך מהרבה בחינות כפי שהוסבר בפוסט זה,

מקווה שנהניתם לקרוא,

חקיקת יוסי

Today I want to talk about Log Shipping.

Log Shipping is an easy way to implement a high availability solution. It uses SQL Server’s transaction log to synchronize a stand by server. The configuration of the log shipping is really easy to set up as I’m about to show.

Basic requirements:

1) The Primary Server must be in “Full” or “Bulk-Logged” recovery model

2) The Secondary Server must be from the same version as the primary (or later) and can be either in “Standby” or “No Recovery” mode. When using the Standby mode, the server can be used for reporting purposes, but it won’t be accessible during the log shipping process.

3) A third monitor server can be used but this is optional.

Step 1 – Configuring the Primary Server:

Right click on the DB you wish to log ship and go to “Tasts -> Ship Transaction Logs…”. Click on “Enable this as a primary database in a log shipping configuration” check box and press on the “Backup Settings…” button.

The “Transaction Log Backup Settings” window will open. In it you will have to specify the path to the folder where the transaction log backups will be saved. Here you can also determine the frequency of transaction log backups (the default is every 15 minutes).

Step 2 – Configuring the Secondary Server:

After the primary server is configured, it’s time to configure the secondary server by pressing the “Add” button. More than one secondary server can be defined to create multiple standby servers.

First, you’ll have to connect to the secondary server.

In the “Initialize Secondary Database” tab you can choose the way you wish to initialize the secondary database (this is pretty self-explanatory).

In the “Copy Files” tab you’ll need to specify the folder where you wish to save the log backups shipped from the primary server, and the frequency of copying between the servers.

In the “Restore Transaction Log” tab you can select the mode you want the secondary server to be in, and determine how often the transaction log will be restored.

As explained in the beginning, choosing the “Standby” mode will allow users to access the secondary DB. This may prevent the log from being restored and fail the log shipping process. When choosing this mode, it’s highly recommended to enable the “Disconnect users” option.

Step 3 – Monitor Server (Optional):

After configuring both servers, you can configure a third, monitor server.

The monitor server keeps track of when the transaction log on the primary database was last backed up, when the secondary servers last copied and restored the backup files, and information about any backup failure alerts.

A single monitor server can monitor multiple log shipping configurations.

You can also generate a script for all of the configurations made to reuse on other servers.

That’s it!!

You’re all done. After pressing “OK” you should see the following window, and the log shipping is up and running.

Created objects:

You might have guessed that the log shipping solution is actually implemented using a series of jobs.

On the primary server you’ll find a job responsible for backing up the log file as specified in the first step.

On the secondary server you’ll find two jobs. The first is responsible for copying the file from the primary to the secondary server, and the second is responsible for restoring the log file as defined in the second step.

On the monitor server you’ll find a job responsible for alerting when no restore has occurred.

You should also find the original log backups on the folder you specified on the primary server, and the copied ones on the secondary.

Fail Over:

There’s no automatic fail over when using log shipping.

Performing the manual fail over involves two simple steps

1) Restoring the secondary server with recovery

2) Disable all the related jobs

Pros and Cons:

I'll start with the good stuff. It’s easy! The default configuration shouldn’t take more than a few minutes to set up. In addition, the standby server can be used for reporting purposes and can be configured on the same instance.

The bad news is the frequent backups add overhead to SQL Server’s operation, and the copying of the backup files takes its toll on the disk subsystem. Another drawback is that fail over must be done manually. If an automatic fail over is important, mirroring might be a more suitable solution.

]]> http://www.madeira.co.il/log-shipping-%e2%80%93-as-easy-as-1-2-3/feed/ 1 http://www.madeira.co.il/the-dedicated-administrator-connection/ http://www.madeira.co.il/the-dedicated-administrator-connection/#comments Mon, 30 Jan 2012 20:53:25 +0000 ‫צ'יקו דרורי‬ http://www.madeira.co.il/?p=4044

Ever been in a crisis that your SQL Server had a spinloop, maxed out resources or severe corruption and  didn’t respond to standard connection requests, and  With pressure mounting to get SQL Server back online you thought that your  last resort was to reboot the server?

This probably because you either haven’t met or forgotten to turn on the DAC your secret backdoor Privilege connection that SQL server offers to us.

DAC? What’s That?

Since SQL Server 2005, Microsoft offers us new feature, the dedicated administrator connection (DAC) a special diagnostic connection for administrators. This diagnostic connection intended to serve as a last means of defense and access SQL Server to execute diagnostic queries and troubleshoot problems even when SQL Server is not responding to standard connection requests as opposed to rebooting the server and causing us long downtime and  possibly data loss, database corruption and/or access violations.

How the DAC works

SQL Server is keeping a backdoor open  resources available just for the DAC, it uses a special reserved scheduler (scheduler_id = 255)  which has one thread for processing requests.

We are all DBA’s and I know that to many of you the thought of abuse thus resources for your maintenance jobs on busy systems cross your mind. Seriously, just don’t go there.That one thread is just one thread, there’s no parallelism for queries running on the DAC. The DAC was not designed for high performance.

some restrictions

Because the DAC exists solely for diagnosing server problems, there are some restrictions on the connection:

Only One Sysadmin Can Ride This Horse At a Time.

SQL Server prohibits running parallel queries or commands with the DAC - For example statements like RESTORE and BACKUP.

Do not use the DAC to run resource-intensive queries - For example a complex join on large table or queries that may block.

Object Explorer Not available for DAC connection - That’s a good thing, we wouldn’t want the power to go to Object Explorer’s head.

Connecting with DAC

By default, the connection is only allowed  from a client running on the local machine. To enable remote clients to utilize the DAC, an option needs to be set which can be done using sp_configure:

   1: exec sp_configure 'remote admin connections', 1 

   2: reconfigure

it’s Highly recommended to enable the ‘remote admin connections’, Especially on clusters configuration. According to BOL servers in cluster configuration and with ‘remote admin connections’ not enable probably Not receive a DAC connection And you can not escape from rebooting the server.

You may need to get firewall ports opened as well, depending on your environment and where you intend to connect from. SQL Server listens for the DAC on TCP port 1434 if available or a TCP port dynamically assigned upon Database Engine startup. The error log contains the port number the DAC is listening on.

To use the DAC you can either access it using SQLCMD from a command prompt:

sqlcmd –A –d master (the -A uses the DAC and the -d connects to the master database)

sqlcmd –d DBname-E –S ServerName\InstanceName –A (the -E uses Windows authentication and the -S connects the server and instance)

Or The more convenient way is using SQL Management Studio with the ADMIN: option when connecting to the server:

- launch SQL Server Management Studio (don't connect to an instance of SQL Server yet)

- select Database Engine Query (icon right next to "New Query")

- put "ADMIN:" in front of the server\instance name

- use your regular Authentication procedures to connect

- select Connect

Note: Object Explorer can’t connect to the DAC. If you open SSMS and have Object Explorer connecting by default, the first connection prompt you see with be for that. If you try to tell that to connect to the DAC, it’ll fail.

conclusion

In previous versions of SQL Server it was often not possible to diagnose a SQL Server instance which was no longer accepting connections. With DAC an administrator has a guaranteed connection to the SQL Server, and is able to resolve problems without the need for a reboot, or restarting the SQL server instance.

For now check whether you have remote admin connections enabled in your environments and try to connect with the DAC in test environment

After my last post about monitoring deadlocks I will now show a way to prevent deadlock and unnecessary locks and to improve system performance.
Introduction:
SQL server has a lot of working mechanisms. The lock mechanism is one of the most important mechanism. Its goal is to keep and secure the data reliability and consistently. This mechanism is very complex and depends on many variables. The mechanism is design to let the system work in the optimal way. It is based on the concept of "all or nothing at all", which means either everything happens during a transaction or nothing happens at all. In specific cases we would like to intervene by setting lock options manually and by doing so to affect the lock mechanism. It is not recommended to use those options on a daily bases, only on specific cases, when we know the type of data that we are dealing with (whether we are receiving data or want to make changes).
The option I am about to present is called "Readpast". Readpast simply means that when we reach for a table data (select, update or delete) and one or more of the table rows or pages are locked by one or more transactions the lock mechanism will ignore those rows and retrieve the rest of the data which is not locked.
The advantage is obviously a performance improvement because there is no waiting time for another transactions locks. But, the disadvantage is more significant than the advantage because even if we accept the loss of data we will not be able to know which part we lost and when.

The most common and recommendable way to use Readpast is when there is a need to implement a queue of command list. For example: there is a table which each row represents a command that needs to be executed. The table also has a separate and an independent process that inserts new commands every time. The order of the commands is not important as long as they are being done. That is why Readpast is the ultimate solution because each process that will turn to the table will receive the next command with no waiting time.

Here is a simple example of how Readpast option impacts the lock mechanism:

· Create a new table with two columns (command ID, command description).

· Populate five new rows.

CREATE TABLE ReadPastTest

)

CommandId INT, CommandDescription VARCHAR(MAX 

(

GO 

INSERT INTO ReadPastTest

 VALUES (1,'do a'),(2,'do b'),(3,'do c'),(4,'do d'),(5,'do e')

 GO

 SELECT CommandDescription

FROM ReadPastTest

GO

· Select the rows.

· Open a new transaction and update one of the rows and keep the transaction open.

BEGIN TRANSACTION 

UPDATE ReadPastTest

SET CommandId = 11

WHERE CommandId = 1

· Select the rows again with another query window that simulate a new process with the Readpast option.

SELECT CommandDescription

(FROM ReadPastTest WITH (READPAST

GO

· Observe the changes.

· Rollback the first transaction and select the table again.

ROLLBACK

SELECT CommandDescription

FROM ReadPastTest

GO

As you can see at the first selection we get all five rows description and the second selection we get only four rows because one row is locked by the first transaction and it was skipped, at last the missing row returns after the rollback.

In conclusion:

the Readpast option is suited to specific cases only, and can provides a perfect solution to those cases. It is important to beware not to use it on every deadlock or locks problems.

Reading this posts title might make us smile: Me helping the optimizer? Isn’t it the one who supposes to help me?

Well, yes and no. The optimizer might help me getting the best execution pal for my queries (as long as I keep performing the right maintenance issues), but is some cases I can hint it to perform even better, and the way to do it is by re-scripting, re-quering and using nothing but some basic TSQL.

As always, an example shown might be worth a thousand words, and my purpose is to make my point with less than a half.

Lets take a look at a script of a procedure which job is to receive data of workers entering and exiting a factory and in addition to inserting it into a table, presenting it on screen for boarder guards:

SET ANSI_NULLS ON
SET QUOTED_IDENTIFIER OFF
GO

CREATE PROCEDURE [dbo].[sp_TLA_Monitor]

—————————
@pUserNo      int      ,
@pSort        smallint ,
@pDoor        int      ,
@pFilterP     smallint = 0 ,
@pFilterV     smallint = 0
—————————

AS
DECLARE @ActLinkString varchar(8000)
SET nocount on

SET @ActLinkString = "

IF @pFilterP > 0 AND @pFilterP < 4
BEGIN
SET @ActLinkString =
'DECLARE @NUMERATOR int '+
'SET @NUMERATOR = ( SELECT MIN( MA.Numerator ) '+
' FROM TLA_OpenEvents MA  '+

CASE WHEN ( @pFilterP = 1 ) THEN 'WHERE DATEDIFF( day   ,  EventTime , GETDATE() ) < 1 ) '
WHEN ( @pFilterP = 2 ) THEN 'WHERE DATEDIFF( Hour  ,  EventTime , GETDATE() ) < '+CAST( @pFilterV  AS VARCHAR) +' ) '
WHEN ( @pFilterP = 3 ) THEN 'WHERE DATEDIFF( day   ,  EventTime , GETDATE() ) < '+CAST( @pFilterV  AS VARCHAR) +' ) '
ELSE ' ' END
END

SET @ActLinkString = @ActLinkString +
'SELECT C.Descript Controller , DE.Descript Device , '+
'cObjCode =  CASE WHEN ISNULL( DO.Code , 0 ) <> 0 THEN DO.Code ELSE CASE WHEN ( MA.Type >= 810 AND MA.Type <= 817 )  THEN MA.ControllerPhisicalID  ELSE MA.Device END END , '+
'cObjType =  CASE WHEN ISNULL( DO.Code , 0 ) <> 0 THEN 1100    ELSE CASE WHEN ( MA.Type >= 810 AND MA.Type <= 817 )  THEN 1200 ELSE 1300 END  END , DO.Descript Door , '+

'MA.Badge Card , W.No WCNo, W.Name WC , DP.No DepNo, DP.Name Dep ,'+
'OE.EventTime , ED.Descript Type ,'+
'OE.FirstOperator ,'+
'FirstOperatorDescr = (SELECT Emp_Name From Emp Where Emp_no = OE.FirstOperator ),'+
'OE.FirstTreatEventTime , '+
'OE.LastOperator  ,'+
'LastOperatorDescr  = (SELECT Emp_Name From Emp Where Emp_no = OE.LastOperator  ),'+
'OE.EndTime , '+
'OE.Numerator , OE.Code , ED.Color , '+
'OE.Severity , OE.Sensitivty ,OE.Priority , OE.Status , '+
'UserType = ISNULL( vwObj.ObjType , 0 ) , '+
'CarNo    = vwObj.ObjCarNo , '+
'EmpNo    = vwObj.ObjNo    ,  Emp_Name = vwObj.ObjName , '+
'xFunction = ( SELECT Descript FROM dvc_def WHERE code = MA.FunctionCode ) , '+
'Pic =  vwObj.ObjPic '+

'FROM TLA_Mov_Access MA '+
'left join vw_ObjectType   vwObj  on vwObj.ObjCode  =  MA.Emp_no AND MA.Tag_Type = vwObj.ObjType '+
'left join Work_c          W      on W.No           =  vwObj.ObjStation '+
'left join Dep             DP     on DP.No          =  vwObj.ObjDep '+
'join TLA_OpenEvents       OE     on OE.Numerator   =  MA.Numerator '+
'left join TLA_Devices     DE     on DE.Code        =  MA.Device '+
'join TLA_EventsDef        ED     on ED.Code        =  MA.Event '+
'left join TLA_Doors       DO     on DO.Code        =  DE.Door '+
'left join TLA_Controllers C      on C.Code         =  MA.ControllerPhisicalID '+
' join dbo.fnc_GetMapPerm('+CAST( @pUserNo AS VARCHAR )+', 12 )  GMP on GMP.Control = ISNULL(DO.Code,-1) '+ –210609

CASE WHEN ( @pFilterP = 0 ) THEN 'WHERE 1 = 1 '
WHEN ( @pFilterP = 1 ) THEN 'WHERE MA.Numerator > @NUMERATOR  '
WHEN ( @pFilterP = 2 ) THEN 'WHERE MA.Numerator > @NUMERATOR  '
WHEN ( @pFilterP = 3 ) THEN 'WHERE MA.Numerator > @NUMERATOR  '
WHEN ( @pFilterP = 4 ) THEN 'WHERE MA.Numerator > (SELECT MAX(Numerator) – '+CAST( @pFilterV  AS VARCHAR)+' FROM TLA_Mov_Access) ' ELSE 'WHERE 1 = 1' END +

CASE WHEN ( @pDoor >= 0 ) THEN ' AND DO.Code = '+CAST( @pDoor AS VARCHAR) ELSE ' ' END +

CASE @pSort
WHEN  0 THEN  ' ORDER BY OE.Numerator DESC '
WHEN  1 THEN  ' ORDER BY OE.Priority DESC '
WHEN  2 THEN  ' ORDER BY OE.Severity DESC '
WHEN  3 THEN  ' ORDER BY OE.Sensitivty DESC ' END

EXEC @ActLinkString

GO
SET QUOTED_IDENTIFIER OFF
GO
SET ANSI_NULLS ON
GO

As you can see, the procedure creates a string to be executed.

We noticed that recently query time of running this procedure is much longer than it used to be – a growth of hundreds of percents. Checking all maintenance issues (indexes, fragmentations etc.) didn’t help the query to run faster like it used to.

Then we ran the Execution Plan of the query and got this:

A kind of a complex one, as a query that uses lots of LEFT JOINs should look like.

Funny thing about it that when we looked at the details we saw in more than one place a huge difference between the estimated and the actual number of rows and between the estimated and the actual number of executions.

Checked again if statistics are updated. Well, they were.

So what is it? What makes the optimizer get the wrong impression? We assumed that if the estimated numbers were closer, much closer in fact, to the actual numbers, optimizer could have probably chosen a second way, different one and perhaps much better one, to execute the query.

Answer is simple: The more complex a query is, the more stages it goes along the way and the more JOINs, mostly of course LEFT JOINs it uses, makes the algorithm become much less accurate. Take for an example a 10% of a standard deviation of the optimizer in estimation at each level. and you got yourself one irrelevant Execution Plan.

So what’s the solution for this problem, you may ask.

A good thing to do would be trying to rescript the query in a way that the optimizer will go step-by-step and perform less complex simultaneous processes.

This query might work better if instead of using one SELECT Statement will all these JOINs, we will try to use temporary tables and devide the SELECT vertically instead of horizontally. In that way, optimizer will use much less JOINs per INSERT INTO and Execution Plan might be devided to more levels, but much more simple in each level.

It should be looking something like this:

SET ANSI_NULLS ON
SET QUOTED_IDENTIFIER OFF
GO

CREATE PROCEDURE [dbo].[sp_TLA_Monitor_Test]

—————————
@pUserNo      int      ,
@pSort        smallint ,
@pDoor        int      ,
@pFilterP     smallint = 0 ,
@pFilterV     smallint = 0
—————————

/*
DOOR       = 1100;
CONTROLLER = 1200;
DEVICE     = 1300;
BUILDING   = 2000;
IMAGE      = 3000;
BUZZER     = 4000;
*/

AS
DECLARE @ActLinkString varchar(8000)
SET nocount on

SET @ActLinkString = "

IF @pFilterP > 0 AND @pFilterP < 4
BEGIN
SET @ActLinkString =
'DECLARE @NUMERATOR int '+
'SET @NUMERATOR = ( SELECT MIN( MA.Numerator ) '+
' FROM TLA_OpenEvents MA  '+

CASE WHEN ( @pFilterP = 1 ) THEN 'WHERE DATEDIFF( day   ,  EventTime , GETDATE() ) < 1 ) '
WHEN ( @pFilterP = 2 ) THEN 'WHERE DATEDIFF( Hour  ,  EventTime , GETDATE() ) < '+CAST( @pFilterV  AS VARCHAR) +' ) '
WHEN ( @pFilterP = 3 ) THEN 'WHERE DATEDIFF( day   ,  EventTime , GETDATE() ) < '+CAST( @pFilterV  AS VARCHAR) +' ) '
ELSE ' ' END
END

SET @ActLinkString = @ActLinkString +
'SELECT '+
'MA.Badge Card, '+
'MA.Tag_Type, '+
'MA.Emp_no, '+
'MA.Numerator Num1, '+
'MA.Event, '+
'MA.Device Dev1, '+
'MA.ControllerPhisicalID, '+
'MA.FunctionCode, '+
'MA.Type Type1, '+
'C.Descript Controller '+
'INTO '+
'#Tmp1 '+
'FROM '+
'TLA_Mov_Access MA '+
'LEFT JOIN TLA_Controllers C on MA.ControllerPhisicalID=C.Code '+

'SELECT '+
'T.Card, T.Tag_Type, T.Emp_no, T.Num1, T.Event, T.Dev1, T.Controller, T.ControllerPhisicalID, T.FunctionCode, T.Type1, '+
'T.UserType, T.CarNo, T.EmpNo, T.Emp_Name, T.Pic, T.ObjStation, T.ObjDep, '+
'OE.EventTime, '+
'OE.FirstOperator,'+
'FirstOperatorDescr = (SELECT Emp_Name From Emp Where T.Emp_no = OE.FirstOperator), '+
'OE.FirstTreatEventTime, '+
'OE.LastOperator, '+
'LastOperatorDescr = (SELECT Emp_Name From Emp Where T.Emp_no = OE.LastOperator), '+
'OE.EndTime, '+
'OE.Numerator, '+
'OE.Code, '+
'OE.Severity, '+
'OE.Sensitivty, '+
'OE.Priority, '+
'OE.Status, '+
'ED.Descript Type, '+
'ED.Color '+
'INTO '+
'#Tmp3 '+
'FROM '+
'#Tmp2 T '+
'JOIN TLA_OpenEvents OE on OE.Numerator=T.Num1 '+
'JOIN TLA_EventsDef ED on ED.Code=T.Event '+

CASE WHEN ( @pFilterP = 0 ) THEN 'WHERE 1 = 1 '
WHEN ( @pFilterP = 1 ) THEN 'WHERE T.Numerator > @NUMERATOR  '
WHEN ( @pFilterP = 2 ) THEN 'WHERE T.Numerator > @NUMERATOR  '
WHEN ( @pFilterP = 3 ) THEN 'WHERE T.Numerator > @NUMERATOR  '
WHEN ( @pFilterP = 4 ) THEN 'WHERE T.Numerator > (SELECT MAX(Numerator) – '+CAST( @pFilterV  AS VARCHAR)+' FROM TLA_Mov_Access) ' ELSE 'WHERE 1 = 1' END +

CASE WHEN ( @pDoor >= 0 ) THEN ' AND T.Code = '+CAST( @pDoor AS VARCHAR) ELSE ' ' END +

CASE @pSort
WHEN  0 THEN  ' ORDER BY T.Numerator DESC '
WHEN  1 THEN  ' ORDER BY T.Priority DESC '
WHEN  2 THEN  ' ORDER BY T.Severity DESC '
WHEN  3 THEN  ' ORDER BY T.Sensitivty DESC ' END

EXECUTE(@ActLinkString)

GO
SET QUOTED_IDENTIFIER OFF
GO
SET ANSI_NULLS ON
GO

And the Execution Plan of it will look something like this:

Try it. You will be surprised to find out how many times it helps queries to be executed much faster. Where it looks like there’s no hope, this solution just might be the one!

So everyone is talking about the next big thing … but what is really the SQL Azure.

SQL Azure provides cloud services to databases that allow you to focus on implementation, rather than build, manage and maintain databases. This means that you no longer need to deal with installing servers, install the SQL, keep big hard disks, servers and so on … all you need to start working you get from Microsoft and now all that's left to you is to focus on developing.

The SQL Azure is part of big family that Microsoft launched on the cloud together with Windows Azure, .NET Services and more.

You can connect to SQL in two ways:

1. By the way we all know and love, through the SSMS.

fiil in the Server name : [server].database.windows.net

2. Via the SQL Azure portal … which I will expand in another post.

So let's talk about two important things … we'll start with the price … The payment is only for what you used, when the costs in relation to ownership or hosting on servers, are significantly lower. And the cost is measured in dollars alone per day. The cost is divided into two parts:
1. Payment for the Database

There are two types of editions that you can purchase web edition and business edition:

Web Edition

$9.99 per database up to 1GB per month

$49.95 per database up to 5GB per month

Business Edition

$99.99 per database up to 10GB per month

$199.98 per database up to 20GB per month

$299.97 per database up to 30GB per month

$399.96 per database up to 40GB per month

$499.95 per database up to 50GB per month

for 100GB and 150GB per month, according to reports from Microsoft, the price shall not exceed more than $499.95.

2. Payments for the Traffic to and from the database, when the Traffic in the cloud there is not charge.

To get the cost for the data transfer you can use the following script:

   1: DECLARE @cost_per_mb DECIMAL (3,2)=0.15

   2: DECLARE @mb INT = 1024*1024;

   3:

   4: SELECT

   5: u.Time_Period,

   6: u.Direction,

   7: @cost_per_mb * u.BandwidthInKB/@mb AS [TotalTransferCost]

   8: FROM (

   9:         SELECT

  10:         Time_Period,

  11:         Direction,

  12:         SUM(Quantity) AS [BandwidthInKB]

  13:         FROM sys.Bandwidth_Usage u

  14:         WHERE

  15:         u.TIME >=DATEADD(D,0-DAY(GetUTCDate()),GetUTCDate())

  16:         AND u.TIME <=EOMONTH(GetUTCDate())

  17:         AND class = 'External'

  18:         AND u.Direction = 'Egress'

  19:         GROUP BY Time_Period, Direction

  20:      ) AS u;

Well, as you already know, you pay for the size of your database, and you pay for the data transfer. To be cost efficient, you really need to design your database correct to avoid redundant data and you need to stop using SELECT *, or including more columns than you need. Otherwise you’ll pay for an overhead that you actually can avoid.

After we understood the price, let's talk a little about the Availability high that Microsoft is so proud of her…

Microsoft promises 99.9% of Accessibility and Availability! And for that When you create a Database, in the background created three copies of your DB on different physical server (not virtual) to prevent data loss if a server falls. One replication is called PRIMARY and the others called SECONDARY, the reads and writes replicated the main base and are delivered in a-synchronic to the others, the secondary, if one replication falls another one rises immediately. The information is considered COMMITED only after the data is written to the primary disk (the transaction log reported that the data is written to the disk) and at least to one of the secondary.

That’s it for this time … hope I helped you educate a little bit

What comes first – The Chicken or the Egg? What happens first – Update Statistics or Re-Compilations ?

The Statistics life cycle

As a regular process of our DB life, data in our tables changes and hence statistics become less accurate. After a certain amount of changes (which I will discuss later in this post), SQL Server marks statistics to be non-valid.

This process marks the statistics as non-valid but doesn’t actually update them. Only when a query is compiled will SQL Server update the relevant statistics (if auto update statistics is on of course). After the statistics are updated, all the queries that rely on these statistics will be marked for re-compilation.

So actually the process is:

1. Many Changes (updates/ inserts/deletes) –> Causes Non Valid Statistics

2. Query Compilation (happens Independently) –> Causes Update Statistics –> Causes All related queries to be re-compiled

When are Statistics marked as non-valid

After how many changes to the table are Statistics marked as non-valid?

After running the following Script you’ll find out that SQL Server marks statistics as non-valid if more than 20% of the rows were changed (counting changes only on columns related to the statistics).

Trace Flag 2371

20% of row changes sounds crazy high if the table has a lot of rows !

Consider for instance, a table containing data from the past 7 years. It would demand to wait a whole 1.4 year of new data to have the statistics updated ! (assuming only inserts are done and same amount of inserts per year). So if you don’t update your statistics manually and have very large tables, you should have a close look at Trace Flag 2371. It actually changes the 20% threshold to be smaller according to the amount of rows in the table. The following chart shows the new thresholds:

The new Trace Flag is currently available on SQL Server 2008 R2 SP1 and on SQL Server 2012.

I recommend reading 2 very interesting posts on Update Statistics:

1. Filtered indexes and filtered stats might become seriously out-of-date (Kimberly L. Tripp)

2. Changes to automatic update statistics in SQL Server – traceflag 2371 (Juergen Thomas)

היו הייתה עיר בשם חלם. חכמי העיר נתקלו באתגר לא פשוט, באחד הגשרים המרכזיים בעיר נפער חור, ורבים מתושבי העיר נפלו מן הגשר ונפצעו. נתכנסו כל חכמי העיר לטכס עצה, אחד מן החכמים שמע על פתרון שיושם בעיר השכנה, וחשב לתומו להעתיק אותו לעיר חלם. הפתרון היה הקמת בית חולים לטיפול בפצועיי הנפילות מהגשר. הדילמה עכשיו הייתה היכן למקם את בית החולים, חשבו וחשבו והגיעו למסקנה שיש למקם את בית החולים בדיוק מתחת לבור שיש בגשר, ככה החולים ייפלו ישר על גג בית החולים ויובהלו לקבלת טיפול רפואי. וכך יצא שהפתרון המזהיר של חכמי העיר למעשה לא הביא לשיפור המצב, ואף גרם להרעה במצב.

הסיפור הנ"ל ממחיש לטעמי בצורה טובה למדיי את כל הקשור לשימוש ב- Locking Hint הנקרא: NOLOCK. ישנה בעיה, בד"כ קשורה לנעילות, ומצד שני ישנו פתרון (NOLOCK) אבל במקרים לא מעטים זה לא הפתרון האמיתי לבעיה אלא פשוט הפתרון הקל. בארגונים רבים ישנו נוהל או מנהג להוסיף לכל שאילתה NOLOCK, בצורה אוטומטית בלי שום קשר לתוכן השליפה וללא מתן תשומת לב לשאלה החשובה כל כך, האם המידע שאני שולף הוא עדכני, תקף ולא עבר שום שינוי בזמן שליפתו! בד"כ הסיבה לשימוש מאסיבי ב- NOLOCK היא זהה במרבית הארגונים, בשלב מסוים, הייתה בעיה של נעילות במערכת שגרמו לעיכובים וכמובן פגעו ישירות בביצועי המערכת, ואז מישהו עלה על הפתרון האולטימטיבי, גם אין נעילות וגם משפרים ביצועים תענוג נכון?? מצטער לקלקל לכם, אבל ממש לא!

ישנן מספר נקודות בסיסיות שעלינו להבין בנוגע לשימוש ב- NOLOCK:

1. Allow Dirty Reads- כאשר מריצים פקודת SELECT עם NOLOCK המשמעות היא שאין שום השפעה לנעילות אחרות, כלומר יכול בהחלט להיווצר מצב שאנו קוראים שורות שבינתיים עשו עליהן ROLLBACK או מצד שני שורות שעדיין לא קיבלו COMMIT. (הערה: NOLOCK עובד בדיוק כמו Read Uncommitted, רק ש-Read Uncommitted מוגדר ברמת ה-Session).

2. לא גורם לנעילות ולא מתחשב בנעילות- וזו בעצם הסיבה שהשימוש ב-NOLOCK משפר את ביצועי המערכת. מחד גיסא הוא איננו מחזיק שום נעילה בעצמו. ומאידך גיסא, הוא איננו מכבד/מתחשב בשום נעילה אפילו לא בנעילה החזקה ביותר Exclusive Lock.

3. רשומות כפולות או היעדר רשומות – את הנקודה החשובה והמעניינת הזו לימד אותי נעם ברזיס. כאשר מרצים שאילתה שגורמת לפעולת Index Scan, יכול להיווצר שבו השימוש ב- NOLOCK יגרום ל-SQL Server לא להציג רשומה מסוימת או יציג אותה פעמיים (בלי שום קשר ל- Dirty Reads). לדוגמא:

משתמש הריץ את השאילתה הבאה, (הטבלה בעלת מספר גדול של רשומות) לטבלה יש אינדקס על עמודת Price:

לאחר 5 שניות ריצה לדוגמא, פעולת  ה- Index Scan הגיעה לערך 15.00 (יש לזכור שהסקירה מתבצעת לפי סדר האינדקס), משתמש אחר ב- Session אחר כמובן, ביצע פעולת INSERT שה- Price הוא 14.00. במצב זה, הרשומה שזה עתה נכנסה, לא תוצג ב- Record Set . משום שפעולת ה- Index Scan דילגה עליו.

אז לפני שאנו מוציאים נוהל חדש בארגון שמחייב להשתמש ב- NOLOCK באופן קבוע, בואו ננסה להבין ממה יכולות להיגרם נעילות רבות (על שלל סוגיהן):

· שימוש לא יעיל ב- Transactions- Transactions צריכות להישמר קצרות ככל הניתן! לשם המחשה, נגיד שיש לנו פרוצדורה ארוכה שבהתחלה מריצים בה פקודת SELECT דיי כבדה, שמכניסה את הנתונים לטבלה זמנית, לאחר מכן, מבצעים חישוב על אותם נתונים, עושים JOIN עם טבלה אחרת ואז בסוף כל זה יש פקודת UPDATE או DELETE. במצב כזה, יש לפתוח את ה- Transaction רק בפקודת ה-UPDATE או ה- DELETE ולא לאורך כל הפרוצדורה ולסגור אותה מיד בסיום הפעולה!

· תכנון לקוי- לעיתים הבעיה נעוצה בסדר הלוגי של הפעולות בתהליך מסוים. יש לנתח ולהבין מה קורה בכל שלב, לנסות לעלות על המצבים בהם יש נעילות שונות. ורק לאחר שמבינים היכן ולמה יש נעילות, לקבל החלטות.

· Foreign Keys – יכולים לגרום לנעילות במקרים מסוימים אבל נושא לפוסט שלם.

מתי נשתמש ב- NOLOCK ומתי לא:

כלל ברזל- אם המידע שאנו שולפים צריך להיות מדויק ואמין וללא חשש שהוא השתנה בזמן ששלפנו אותו אז לעולם לא נשתמש ב- NOLOCK. לדוגמא, דוחות שקשורים לכספים וחיובים של לקוחות או גביית כסף, לעולם לא נפיק אותם עם NOLOCK (זו רק דוגמא להמחשה ישנם עוד מקרים רבים אחרים).לעומת זאת, לעיתים תכופות מריצים שאילתות שונות כלליות מאד כדי לקבל תמונת מצב ראשונית על מבנה של טבלאות ועל הנתונים שיש בהן, בדיקות כאלו לרוב לא מחייבות שהמידע יהיה עקבי לאותה שנייה שבא קראנו אותו. יתר על כן, לעיתים ישנם דוחות שמטרתם להציג אומדן ולא נתונים מדויקים, גם במקרים כאלו אין חשש להשתמש ב-NOLOCK.

לסיכום, כמו בהרבה תחומים אחרים ב- SQL Server אנחנו במצב מתמיד של שמיכה קצרה. שמתם NOLOCK הרווחתם ביצועים אבל להבין שיש מחיר לכך, אין ארוחות חינם, נכון, הרווחנו ביצועים, אבל הפסדנו דבר חשוב לא פחות והוא עקביות ואמינות המידע. אז בפעם הבאה שאתם מוסיפים NOLOCK, תעצרו לשנייה ותחשבו על ההשלכות לחיוב ושלילה ותקבלו החלטה מושכלת ומתוך מודעות.

חקיקת יוסי

Today I chose to talk about Virtual Log Files or VLF in short. VLFs are the physical files that implement the log file. The log file’s log records are stored sequentially in a set of these physical files.

Creating a VLF:

Every time space is allocated for the log file (Initial creation or log growth) new VLFs are created behind the scenes. The number of new VLFs is determined by the amount of space allocated.

1) 0MB > Space Added >= 64MB ,  4 new VLFs

2) 64MB > Space Added >= 1GB ,  8 new VLFs

3) 1GB > Space Added,                 16 new VLFs

To demonstrate, I’ll use this script to create a DB with a log file in various sizes and auto growths. The database’s data file sizes is irrelevant.

USE
    master;
GO

IF DB_ID('MyDB') IS NOT NULL
BEGIN
    DROP DATABASE MyDB;
END;

CREATE DATABASE MyDB
ON (
    NAME = MyDB,
    FILENAME = 'C:\Program Files\Microsoft SQL Server\MSSQL10_50.SQLSERVERR2_SHAY\MSSQL\DATA\MyDB.mdf',
    SIZE = 10MB,
    FILEGROWTH = 1MB)
LOG ON (
    NAME = MyDB_Log,
    FILENAME = 'C:\Program Files\Microsoft SQL Server\MSSQL10_50.SQLSERVERR2_SHAY\MSSQL\DATA\MyDB_Log.ldf',
    SIZE = 10MB, -- 10MB, 250MB, 65MB, 1025MB
    FILEGROWTH = 6MB);
GO

To view the size and number of the VLFs created I’ll use sys.database_files and DBCC LOGINFO.

USE
    MyDB;
GO

SELECT
    File_Name    = name ,
    Size        = CAST(size*8/1024 AS VARCHAR(10))+'MB' , -- size is in pages (8KB)
    Auto_Growth    = CASE is_percent_growth
                    WHEN 1 THEN CAST(growth AS VARCHAR(10))+'%'
                    ELSE CAST(growth*8/1024 AS VARCHAR(10))+'MB'
                END
FROM sys.database_files
WHERE type_desc = 'LOG';

DBCC LOGINFO;
GO

10MB:

Our log file is 10MB, and as expected, we have 4 VLF files. But if we sum the size of the files we’ll get to a total of 10,477,568 Bytes while 10MB are 10,485,760 Bytes. There are 8,192 Bytes (size of a page) missing. This space is used by the file header and it does not store log records. You can see it in the StartOffset for the first VLF. This is true for all log file sizes.

Here are the results for bigger log files:

65MB:

250MB:

1025MB:

Ok, so now we know how VLFs are created, but before we continue, here’s a short overview on DBCC LOGINFO.

DBCC LOGINFO:

The fields that will be relevant to better understand the VLF behavior are:

FileId – If you have one log file, this number will be the same

FileSize – This number is in Bytes

StartOffset – This number is in Bytes

FSeqNo – The file sequence number. The VLF with the highest FSeqNo is the one where current log records are being written into. FSeqNo = 0 indicates that the file hasn’t been used yet.

Status – Two possible values: 0 – VLF is recyclable, 2 – VLF may be active (Highest FSeqNo must be active)

Parity – Two possible values: 64 and 128. Every time a VLF is reused, the parity value is switched.

CreateLSN – Indicates at what LNS (Log Sequence Number) the VLF was created. CreateLSN = 0 states that the VLF was created at the time the log was created. Files with the same CreateLSN were created at the same time.

To see how the VLFs are being written on, I’ll create some activity in the DB.

The DB is set for simple recovery mode for now.

USE
    MyDB;
GO

ALTER DATABASE MyDB
SET RECOVERY SIMPLE;
GO

CREATE TABLE dbo.MyTable (Value SYSNAME);

INSERT INTO dbo.MyTable
SELECT TOP 1000 T1.name
FROM sys.columns AS T1
CROSS JOIN sys.columns AS T2;
GO 20

The empty VLF files will be filled one by one until they are all full.

Notice that the first 2 VLF statuses are 0. This means that these VLFs are recyclable and can be written on.

After generating more activity the first VLF has been overwritten and its FSeqNo changed from 22 to 26. Now the 3 other VLFs are recyclable and have status = 0.

Log Truncation:

Truncation is when inactive VLFs are marked as recyclable. This is a logical operation. Truncation occurs in the following events:

1) When a checkpoint occurs (only in auto truncate mode)

2) When the log is backed up (not in COPY_ONLY)

3) When the recovery mode is set to SIMPLE

Auto Truncate Mode:

In Auto Truncate VLFs are being overwritten once they become inactive. The log file is in this mode when the DB is in SIMPLE recovery mode, or in FULL but no backup has been made.

Adding new VLFs (Growing the Log):

In order to make the log file grow and create more VLFs we must first get out if the Auto Truncate mode. To do that, we’ll set the DB to FULL recovery mode and back it up.

ALTER DATABASE MyDB
SET RECOVERY FULL;
GO

BACKUP DATABASE MyDB
TO DISK = 'C:\Users\Shay\Documents\BackUp\MyDB_BU.bak';
GO

Once we generate some more activity, all of the VLFs are full and new ones are created.

The log file grew by 6MB and 4 new VLFs are added to it. All of the files that have already been written on (FSeqNo > 0) are marked as active (Status = 2) and cannot be truncated.

Forcing Truncation:

In order to be able to recycle the VLFs, we’ll buck up the Log File.

BACKUP LOG MyDB
TO DISK = 'C:\Users\Shay\Documents\BackUp\MyDB_LOGBU.bak';
GO

After it has been bucked up, all of the VLFs are marked as inactive (Status = 0) except for the VLF that is being used at the moment. Note that the log file hasn’t shrunk, and is still 16MB.

Now new log records can be written on the truncated VLFs.

Deleting VLFs (Shrinking the Log):

If the log file is too big, it can be shrunk by removing inactive VLFs from it. To do that we’ll make the DB go into Auto Truncate mode by set the recovery mode back to SIMPLE. If we are already in SIMPLE recovery mode, we can force a truncation by using the CHECKPOINT command.

ALTER DATABASE MyDB
SET RECOVERY SIMPLE;
GO

CHECKPOINT;
GO

Now we can shrink the log file

DBCC SHRINKFILE(2,5); -- File Id, File Size in MB
GO

The file has shrunk to 6MB instead of 5 like we wanted. The reason is not all of the inactive VLFs are being removed during the file shrinking. In addition, the size of a single VLF can never change.

Now What?

Now after we have a better understanding of how VLFs are created, truncated and deleted, and have also learned how to control these events, we as DBAs can modify our database for better performance by controlling the number of VFLs in the log file.

But what do we want to achieve?

First, let’s consider our options:

Many VLFs:

Having a lot of relatively small VLFs means that the log file has more flexibility during normal activity, but may cause problems during recovery and restore, and also during CHECKPOINT when in Auto Truncate mode. All of the inactive VLFs are being inspected at these events, and having a lot of then may be time consuming.

In SQL Server 2012 an error log message will appear when going over 1,000 VLFs.

Few VLFs:

Having only a few big VLFs could cause problems involving truncation. A small amount of log records can prevent the VLF from being truncated. Shrinking the log file might suffer the same problems.

Bottom Line:

There is no conclusive best Practice when it comes to determining the desired number of VLF files in the log. DBAs who have tested the subject report the “Sweet Spot” is around 100 VLFs. Like every other aspect of DB design, you should test it on your own system and configure it for what suits you best.

If you have tested it already, I'd like it if you could share your insights.

One of the coolest and fantastic new features of SQL Server 2008 is the new data type HierarchyID. You’ve probably been confronted with a scenario in which you needed to data tree of some sort, and you’ve probably struggled with joins, recursive operations, and other strategies in the past. With this new type you can implement tree structures with ease, making use of the best indexing strategy.

The HierarchyID data type allows you to construct relationships among data elements within a table, specifically to represent a position in a hierarchy.

Some Facts:

-New system CLR type supporting trees.

-Extremely compact – Internally stored as varbinary <= 900 bytes.

-Easy to understand – Holds a path that provides a topological sort of a tree.

-Has a set of methods that provide tree functionality: GetAncestor, GetDescendant,  GetLevel, GetRoot, IsDescendant, Parse, Read, Reparent, ToString, Write.

-Efficient querying that can utilize depth-first and breadth-first indexes.

The hierarchyid data type has the following limitations:

-A column of type hierarchyid does not automatically represent a  tree. It is up to the application to generate and assign hierarchyid values in such a way that the desired relationship between rows is reflected in the values. Some applications might not even want to have a column of type hierarchyid represent a tree. Perhaps the values are references to location in a hierarchy defined in another table.

-It is up to the application to manage concurrency in generating and assigning hierarchyid values. There is no guarantee that hierarchyid values in a column are unique unless the application uses a unique key constraint or enforces uniqueness itself through its own logic.

-Hierarchical relationships represented by hierarchyid values are not enforced like a foreign key relationship. It is possible and sometimes appropriate to have a hierarchical relationship where A has a child B, and then A is deleted leaving B with a relationship to a nonexistent record. If this behavior is unacceptable, the application must query for descendants before deleting parents.

Methods provided are as follows:

GetAncestor: Returns a HierarchyID that represents the n^th ancestor of this HierarchyID node.

GetDescendant: Returns a child node of this HierarchyID node.

GetLevel: Returns an integer that represents the depth of this HierarchyID node in the overall hierarchy.

GetRoot: Returns the root HierarchyID node of this hierarchy tree. Static.

IsDescendant: Returns true if the passed-in child node is a descendant of this HierarchyID node.

Parse: Converts a string representation of a hierarchy to a HierarchyID value. Static.

Reparent: Moves a node of a hierarchy to a new location within the hierarchy.

ToString: Returns a string that contains the logical representation of this HierarchyID.

Indexing Strategies :

There are two strategies for indexing hierarchical data:

-Depth-first

A depth-first index, rows in a subtree are stored near each other. For example, all employees that report through a manager are stored near their managers' record- Useful for querying subtrees.

-Breadth-first

A breadth-first stores the rows each level of the hierarchy together. For example, the records of employees who directly report to the same manager are stored near each other – Useful for querying immediate children.

Everyone found that genius but a lot broke their teeth using it in real life so let's 

Try it:

Let's assume we already have the following products 

The Idea is to represent a product tree:

First, we need to create a table to store the product hierarchy:

   1: CREATE TABLE dbo.ProductTree

   2: (

   3: [hierarchyid] [hierarchyid] NOT NULL,

   4: [Node] [nvarchar](4000) NOT NULL, --String representation of the HierarchyID.

   5: [ProductID] [int] NOT NULL

   6: )ON [PRIMARY]

   7: GO

   8: CREATE UNIQUE CLUSTERED INDEX [IX_ProductTree] ON dbo.ProductTree

   9: (

  10:  [hierarchyid],

  11:  [ProductID]

  12: )

  13: GO

Let's start building the tree! First, the root: 

   1: --Insert The Root -> Product

   2: INSERT INTO dbo.ProductTree

   3: (hierarchyid, Node, ProductID)

   4: VALUES

   5: (hierarchyid::GetRoot(), hierarchyid::GetRoot().ToString(), 1)

   6: GO

Notice the call GetRoot() to get the root node reference.

So now we have the root node, now we need to add the first child:

   1: --Insert First Child(IPhone)

   2: DECLARE @ProductID INT = 2 -- The First Child  = iPhone

   3: DECLARE @hierarchyid HIERARCHYID

   4: DECLARE @Parenthierarchyid HIERARCHYID

   5:

   6: --Get Parent hierarchyid

   7: SELECT @Parenthierarchyid = hierarchyid

   8: FROM dbo.ProductTree

   9: WHERE ProductID = 1

  10:

  11: --Get the hierarchyid

  12: SELECT @hierarchyid = @Parenthierarchyid.GetDescendant(NULL,NULL)

  13: FROM dbo.ProductTree

  14:

  15: --Insert First Child(IPhone)

  16: INSERT INTO dbo.ProductTree

  17: (hierarchyid, Node, ProductID)

  18: VALUES

  19: (@hierarchyid, @hierarchyid.ToString(), @ProductID)

  20:  Go

Notice the call GetDescendant() to get the descendant of a given node. 

It has a great significance in terms of finding the new descendant position 

Get the descendants etc.

We'll be inserting the "Mac", in the same level, below "Product" 

And next to "iPhone".

   1: --Insert Second Child(Mac)

   2: DECLARE @ProductID INT = 3 -- The Second Child = Mac

   3: DECLARE @FirstChildhierarchyid HIERARCHYID

   4: DECLARE @hierarchyid HIERARCHYID

   5: DECLARE @Parenthierarchyid HIERARCHYID

   6: --Get First Child hierarchyid and Parent hierarchyid

   7: SELECT @FirstChildhierarchyid = hierarchyid,

   8:        @Parenthierarchyid = hierarchyid.GetAncestor(hierarchyid.GetLevel())

   9: FROM dbo.ProductTree

  10: WHERE ProductID = 2 -- iPhone

  11: --Get the hierarchyid

  12: SELECT @hierarchyid = @Parenthierarchyid.GetDescendant(@FirstChildhierarchyid,NULL)

  13: FROM dbo.ProductTree

  14: --Insert Second Child(Mac)

  15: INSERT INTO dbo.ProductTree

  16: (hierarchyid, Node, ProductID)

  17: VALUES

  18: (@hierarchyid, @hierarchyid.ToString(), @ProductID)

  19:

  20: Go

Notice the call GetAncestor() to get the ancestor 

Of a given child node by the level (int).

Since we needed to add a new product,

At the same level of the "iPhone" product,

we had to get a node reference next to it, hence the @Parenthierarchyid.GetDescendant(@FirstChildhierarchyid,NULL).

Now let's add a second level child to "iPhone":

   1: --Insert Second Level Child(iPhone 4S)

   2: DECLARE @ProductID INT = 4 -- The Second Level Child = iPhone 4S

   3: DECLARE @hierarchyid HIERARCHYID

   4: DECLARE @Parenthierarchyid HIERARCHYID

   5: --Get the Parent hierarchyid

   6: SELECT @Parenthierarchyid = hierarchyid

   7: FROM dbo.ProductTree

   8: WHERE ProductID = 2 -- iPhone

   9: --Get the hierarchyid

  10: SELECT @hierarchyid = @Parenthierarchyid.GetDescendant(NULL,NULL)

  11: FROM dbo.ProductTree

  12: --Insert Second Level Child(iPhone 4S)

  13: INSERT INTO dbo.ProductTree

  14: (hierarchyid, Node, ProductID)

  15: VALUES

  16: (@hierarchyid, @hierarchyid.ToString(), @ProductID)

  17: Go

Here is the result:

?Notice the path representation and how it all fits in. Pretty neat huh
Conclusion

According performances, you can use the HierarchyID type without hesitation when modeling tree structures in a relational database. That new type fulfills its goal although it's not simple to implement…

...Hope you like this article I will write more about HierarchyID very soon