One of the basic rules when creating or altering a table is to configure and adjust the right columns properties such as data types and nullable values. It is done because we assume what kind of data those columns will contain.
For example, we will not define a person's age column with "int" data type because it will just waste unnecessary storage space.
Even if we will configure the optimal data types for each column we will still face columns that holds a lot of nullable or zeroes values. Those will use storage just as if they were containing with data.
SQL server 2008 offers a new column property which is called "sparse columns".
The goal of sparse columns is to minimize storage ignoring empty columns cells.
Obviously, nothing comes free. Every sparse column data will increase each column's data storage use by four bytes. Therefore if you define a column with the data type "int" that normally uses four bytes it will now need eight bytes.

Best practice recommendations:

The main idea of this post is to use "spares columns" only when the nullable percentage of data is significantly high (Microsoft numbers are between 20-40 percent).
Secondly, use "sparse columns" when dealing with big storage data types because then the addition of the four bytes will be less significant comparing small data types such as "bit" or "tinyint".
Third, when we want to create a filtered index on a column that holds nullable values, the best practice is to create the column with sparse. That way the SQL server knows to ignore the nullable values and save precious storage.

Pros

· Reading and writing on "sparse columns" is no different than regular columns.
· We can define an additional xml "column set" that will point to the "sparse columns" and gather their data (see an example at the end of the   post).

Cons

· "Sparse columns" is a property of the storage layer and not apart of the table properties layer so when we perform a "select into" command     the sparse property will not be copied to the new table.
· There are several data types that do not support "sparse column" such as:
Text, Timestamp, Geometry, image, ntext, user-defined data types.
· "Sparse columns" cannot be configured on "file stream" columns.
· "Sparse columns" cannot be configured on a computed columns.
· "Sparse columns" cannot have default values.
· "Sparse columns" cannot be configured on clustered index columns.

Examples

In order to present and examine the use of "sparse columns" I created two identical tables. One with "sparse columns" and the other without "sparse columns".
Then I inserted into the two tables 50,000 rows with nullable values.
Then, I called the "sp_spaceused" procedure to see the differences between the results. It is easy to see that the table with the "sparse columns" used much less storage space.

CREATE TABLE
UnSparsed_Table(
ID INT IDENTITY(1,1),
ColX INT,
ColY VARCHAR(100),
ColZ DateTime)
GO

CREATE TABLE
Sparsed_Table(ID INT IDENTITY(1,1),
ColX INT SPARSE,
Coly VARCHAR(100) SPARSE,
ColZ DateTime SPARSE)
GO

DECLARE
@iterator INT = 0
WHILE @iterator < 50000
BEGIN
    INSERT INTO
    UnSparsed_Table VALUES (NULL,NULL, NULL)
    INSERT INTO
    Sparsed_Table VALUES (NULL, NULL, NULL)
    SET
    @iterator+=1
END
GO

sp_spaceused
'UnSparsed_Table'
GO

sp_spaceused�
'Sparsed_Table'
GO

DROP TABLE
UnSparsed_Table
GO

DROP
TABLE Sparsed_Table
GO

In order to present and examine the use of "column set" I created a table with two "sparse columns" and another XML column. Then I inserted one row into the table and selected it to see the results. As you can see the XML column holds the "sparse columns" data.

CREATE TABLE Example2(
ColI int SPARSE,
ColJ int SPARSE,
XMLC xml column_set FOR ALL_SPARSE_COLUMNS)
GO

INSERT Example2(ColI, ColJ)
VALUES (1,2)
GO

SELECT XMLC FROM Example2
GO

For conclusion, "sparse columns" is a great property when it comes to saving storage space, but remember to use it wisely.

Hope you enjoyed reading my post.
Tomer.

At first, let's start with a cool relaxation exercise. Close your eyes and imagine that your DBA work environment includes only one server. Nice, isn't it? Now let's wake up to real life. As you all know typical, small, simple environment will include about 10 servers and in big companies it can reach to thousands or more servers (I can't imagine how many Microsoft has). Therefor raises the need to monitor, manage and organize all servers.

Examples of organized group of servers can be by their work environment (dev, qa, production), "SQL Server" versions and even by geographic spreading.

As part of the DBA work routine we would like to collect info to our servers on a daily basis. We can do it by connecting to each one of server separately or by third part applications, but that can be very inconvenient and can take a lot of time.

It would be very nice if in our basic tool the "management studio" we can execute one command and "SQL Server" will connect to each server and collect all the data.

It would also be nice if it will combine all the data in one table with a new "server's name" column.

Since 2008 it can be done, and it's called "Central management servers", also known as CMS. With that new feature we can create groups of servers from all kinds of "SQL servers" versions (old and newer). You have to choose one server (2008 version) to be the central management server and it will hold the list of the server's groups. The list will be available for other users for as long as the server service is running. That way we will be able to query each of the groups.

In order to setup CMS there are few requirements:

- You will need at least one instance of "SQL Server" 2008 that will be the management server.

- All the connection to the servers should be with windows authentication because CMS cannot hold the connection details for SQL authentication.

- The server that was defined to manage the list of servers cannot participate in the registered servers.

This is how to create CMS:

"BOL" manual

1. In SQL Server Management Studio, on the View menu, click Registered Servers.

2. In Registered Servers, expand Database Engine, right-click Central Management Servers, point to New, and then click Central Management Servers.

3. In the New Server Registration dialog box, register the instance of SQL Server that you want to become the central management server.

4. In Registered Servers, right-click the central management server, point to New, and then click New Server Group.

5. Type a group name and description, and then click OK.In Registered Servers, right-click the central management server group, and then click New Server Registration.

6. In the New Server Registration dialog box, register one or more instances of SQL Server that you want to become members of the server group.

And here is an example of CMS creation:

Here are some examples for queries on multiple servers:

Limitations and cons:

- It is impossible to save the table data because each row represent different server. If you try to insert the rows to another table it will create separate table for each server with only its rows.

- You cannot join the data with other tables.

- You cannot sort the data. The order by command will sort the rows by each server.

- "SQL Server" connects to each server and run your commands. If one of the servers is "case sensitive" and your commend is not, then you will get an error.

- You must confirm that all servers return the same number of columns. If that is not the case, you can change the setting of CMS to return separated tables instead of one big table.

Note about security:
As you imagine, if you don’t have access to read the data of one of the servers you will not be able to run query on a server that is listed in the group.

For conclusion, CMS can save you a lot of expansive time and make your multiple servers much more manageable and easy to use and monitor.

Tomer Shtrum.

After my last post about monitoring deadlocks I will now show a way to prevent deadlock and unnecessary locks and to improve system performance.
Introduction:
SQL server has a lot of working mechanisms. The lock mechanism is one of the most important mechanism. Its goal is to keep and secure the data reliability and consistently. This mechanism is very complex and depends on many variables. The mechanism is design to let the system work in the optimal way. It is based on the concept of "all or nothing at all", which means either everything happens during a transaction or nothing happens at all. In specific cases we would like to intervene by setting lock options manually and by doing so to affect the lock mechanism. It is not recommended to use those options on a daily bases, only on specific cases, when we know the type of data that we are dealing with (whether we are receiving data or want to make changes).
The option I am about to present is called "Readpast". Readpast simply means that when we reach for a table data (select, update or delete) and one or more of the table rows or pages are locked by one or more transactions the lock mechanism will ignore those rows and retrieve the rest of the data which is not locked.
The advantage is obviously a performance improvement because there is no waiting time for another transactions locks. But, the disadvantage is more significant than the advantage because even if we accept the loss of data we will not be able to know which part we lost and when.

The most common and recommendable way to use Readpast is when there is a need to implement a queue of command list. For example: there is a table which each row represents a command that needs to be executed. The table also has a separate and an independent process that inserts new commands every time. The order of the commands is not important as long as they are being done. That is why Readpast is the ultimate solution because each process that will turn to the table will receive the next command with no waiting time.

Here is a simple example of how Readpast option impacts the lock mechanism:

· Create a new table with two columns (command ID, command description).

· Populate five new rows.

CREATE TABLE ReadPastTest

)

CommandId INT, CommandDescription VARCHAR(MAX 

(

GO 

INSERT INTO ReadPastTest

 VALUES (1,'do a'),(2,'do b'),(3,'do c'),(4,'do d'),(5,'do e')

 GO

 SELECT CommandDescription

FROM ReadPastTest

GO

· Select the rows.

· Open a new transaction and update one of the rows and keep the transaction open.

BEGIN TRANSACTION 

UPDATE ReadPastTest

SET CommandId = 11

WHERE CommandId = 1

· Select the rows again with another query window that simulate a new process with the Readpast option.

SELECT CommandDescription

(FROM ReadPastTest WITH (READPAST

GO

· Observe the changes.

· Rollback the first transaction and select the table again.

ROLLBACK

SELECT CommandDescription

FROM ReadPastTest

GO

As you can see at the first selection we get all five rows description and the second selection we get only four rows because one row is locked by the first transaction and it was skipped, at last the missing row returns after the rollback.

In conclusion:

the Readpast option is suited to specific cases only, and can provides a perfect solution to those cases. It is important to beware not to use it on every deadlock or locks problems.

In order to monitor and analyze deadlocks in your server you will first need to understand what does deadlock mean.

Books on line definition:

"A deadlock occurs when two or more tasks permanently block each other by each task having a lock on a resource which the other tasks are trying to lock."

In other words, this is a situation when two or more processes holds a lock on a different resource try to get a lock on the other process resource which is already locked. This will eventually create an endless loop. Sql server knows how to deal with those kind of situations with the "Deadlock detection" mechanism.

The mechanism continuously (every five seconds) checks the lock monitor thread and searches for deadlocks.

If a deadlock occurs the mechanism choses a process "victim" and rolls back his transaction. The victim is the process that its rollback will need the less resources.

Note – If we want we can predefine the process importance level and then the victim will be chosen by our definition.

Rollback the victim process will enable the other process to finish its transaction and that will end the loop.

It will be wise to create a "try and catch" mechanism in the processes transactions that will re-execute the victim process after a few random seconds.

Here is a script as an example of a simple and classical deadlock, which contains two transactions for each query window.

first query:

Transaction A--

USE AdventureWorks

GO

BEGIN TRANSACTION

Statement 1--

UPDATE Sales.SalesOrderDetail

SET OrderQty = OrderQty * 2

WHERE SalesOrderID = 43659 and SalesOrderDetailID = 1

Hold for 10 seconds--

WAITFOR DELAY '00:00:10'

Statement 2--

SELECT * FROM HumanResources.Department

WHERE DepartmentID = 1

COMMIT

GO

second query:

    USE AdventureWorks

   GO

   BEGIN TRANSACTION

   Statement 1--

    UPDATE HumanResources.Department

   SET Name = Name + ' added text'

   WHERE DepartmentID = 1

   Hold for 10 seconds--

   WAITFOR DELAY '00:00:10'

  Statement 2--

   SELECT * FROM Sales.SalesOrderDetail

   WHERE SalesOrderID = 43659 and SalesOrderDetailID = 1

   COMMIT

   GO

The scripts runs on AdvantureWorks database.

Script flow:

1. Open two queries with two transactions (each query resemble each process).

2. Each process updates different table.

3. Each process waits ten seconds (in order to execute the other query).

4. Each process tries to select the other process table.

5. You will be able to see in the end that one process finished its transaction and the other one will print a massage that a deadlock occurred.

Now that we understand what is a deadlock, these are the main tools to monitor and identify deadlocks:

Windows performance monitor.

This tool allows you to see if a deadlock occurs on your server.

In order to start we need to open "windows performance monitor" and to add the "deadlock\sec" counter in the "sql server: locks category". All is left to do is run again the two script queries that creates the deadlock or wait for a deadlock to occur on the server. We will be able to see the changes on the graph.

Graph example:

Sql server trace flags:

You can monitor and analyze deadlocks with two sql server trace flags.

Trace flag 1204 and trace flag 1222. They are similar and will write the deadlock description to the sql server error logs. The difference between them is that trace flag 1204 reports for each process separately and trace flag 1222 reports firstly on the processes and then on the resources. It is possible to turn both flags on by typing the command:

"DBCC TRACEON (trace number)" for each trace flag.

According to the explanation:

1. turn on both trace flag

2. execute both scripts

3. open sql server error logs to see which processes were participate in the deadlock, who was chosen to be the victim and most important, what is the transactions code for future analyze.

Error log example:

Sql server profiler:

The sql server profiler lets you monitor almost everything that occurs on your server including deadlocks. In my opinion this is the easiest and most efficient tool to monitor and analyze deadlocks. With this tool we are able to see in a shape of a graph:

1. Which processes participate in the deadlock.

2. Which resources they already locked.

3. Which resource they are trying to lock.

4. Who was chosen to be the victim.

5. Transactions code.

All we got to do is:

1. Open the sql server profiler.

2. On the "general" tab at "use the template" row choose "TSQL_Locks".

3. On the "Event Selection" tab mark only the "deadlock Graph" under locks section.

4. Run the trace and wait for the deadlock to occur.

Sql server profile example:

Note – It is important to remember to turn off all the traces you used after you finish because they use a lot of resources.

For conclusion, now that you know how to identify and monitor your server deadlocks its time to dig into the processes themselves and try to figure how to re-write or synchronize them in order to avoid any unnecessary deadlocks on your server.

Shtrum Tomer.