SQL SERVER – Introduction to SQL Server Encryption and Symmetric Key Encryption Tutorial with Script

SQL Server 2005 and SQL Server 2008 provide encryption as a new feature to protect data against hackers’ attacks. Hackers might be able to penetrate the database or tables, but owing to encryption they would not be able to understand the data or make use of it. Nowadays, it has become imperative to encrypt crucial security-related data while storing in the database as well as during transmission across a network between the client and the server.

Encryption hierarchy is marked by three-level security. These three levels provide different mechanisms for securing data across networks and local servers. Different levels of hierarchies allow multiple instances of services (e.g., SQL Server Services) to run on one physical server.

  • Windows Level – Highest Level – Uses Windows DP API for encryption
  • SQL Server Level - Moderate Level – Uses Services Master Key for encryption
  • Database Level – Lower Level – Uses Database Master Key for encryption

There are two  kinds of keys used in encryption:

  • Symmetric Key – In Symmetric cryptography system, the sender and the receiver of a message share a single, common key that is used to encrypt and decrypt the message. This is relatively easy to implement, and both the sender and the receiver can encrypt or decrypt the messages.
  • Asymmetric Key – Asymmetric cryptography, also known as Public-key cryptography, is a system in which the sender and the receiver of a message have a pair of cryptographic keys – a public key and a private key – to encrypt and decrypt the message. This is a relatively complex system where the sender can use his key to encrypt the message but he cannot decrypt it. The receiver, on the other hand, can use his key to decrypt the message but he cannot encrypt it. This intricacy has turned it into a resource-intensive process.

Yet another way to encrypt data is through certificates. A public key certificate is a digitally signed statement that binds the value of a public key to the identity of the person, device, or service that holds the corresponding private key. A Certification Authority (CA) issues and signs certifications. Download complete script here.

Please create a sample database that we will be use for testing Encryption. There are two different kinds of encryptions available in SQL Server:

  • Database Level – This level secures all the data in a database. However, every time data is written or read from database, the whole database needs to be decrypted. This is a very resource-intensive process and not a practical solution.
  • Column (or Row) Level – This level of encryption is the most preferred method. Here, only columns containing important data should be encrypted; this will result in lower CPU load compared with the whole database level encryption. If a column is used as a primary key or used in comparison clauses (WHERE clauses, JOIN conditions) the database will have to decrypt the whole column to perform operations involving those columns.

Let’s go over a simple instance that demonstrates the encryption and the decryption process executed with Symmetric Key and Triple DES encryption algorithm.

/* Create Database  */
USE master
GO
CREATE DATABASE EncryptTest
ON PRIMARY ( NAME = N'EncryptTest', FILENAME = N'C:\EncryptTest.mdf')
LOG ON ( NAME = N'EncryptTest_log', FILENAME = N'C:\EncryptTest_log.ldf')
GO

First, let’s create a sample table and then populate it with sample data. We will now encrypt one of the two columns of the table.

/* Create table and insert data in the table */
USE EncryptTest
GO
CREATE TABLE TestTable (FirstCol INT, SecondCol VARCHAR(50))
GO
INSERT INTO TestTable (FirstCol, SecondCol)
SELECT 1,'First'
UNION ALL
SELECT 2,'Second'
UNION ALL
SELECT 3,'Third'
UNION ALL
SELECT 4,'Fourth'
UNION ALL
SELECT 5,'Fifth'
GO
/* Check the content of the TestTable */
USE EncryptTest
GO
SELECT *
FROM TestTable
GO

The preceding code will return the result depicted in the subsequent figure.

Result of the SQL query

Every database can have one master key. Database master key is a symmetric key used to protect the private keys of certificates and asymmetric keys present in the database. It uses Triple DES algorithm together with user-provided password to encrypt the keys.

/* Create Database Master Key */
USE EncryptTest
GO
CREATE MASTER KEY ENCRYPTION
BY PASSWORD = 'SQLAuthority'
GO

Certificates are used to safeguard encryption keys, which are used to encrypt data in the database. SQL Server 2005 has the capability to generate self-signed X.509 certificates.

/* Create Encryption Certificate */
USE EncryptTest
GO
CREATE CERTIFICATE EncryptTestCert
WITH SUBJECT = 'SQLAuthority'
GO

The symmetric key can be encrypted by using various options such as certificate, password, symmetric key, and asymmetric key. A number of different algorithms can be employed for encrypting key. The supported algorithms are DES, TRIPLE_DES, RC2, RC4, RC4_128, DESX, AES_128, AES_192, and AES_256.

/* Create Symmetric Key */
USE EncryptTest
GO
CREATE SYMMETRIC KEY TestTableKey
WITH ALGORITHM = TRIPLE_DES ENCRYPTION
BY CERTIFICATE EncryptTestCert
GO

Now add a column of type varbinary to the original table, which will store the encrypted value for the SecondCol.

/*  Encrypt Data using Key and Certificate
Add Columns which will hold the encrypted data in binary */
USE EncryptTest
GO
ALTER TABLE TestTable
ADD EncryptSecondCol VARBINARY(256)
GO

Before the key is used, it needs to be decrypted using the same method that was used for encrypting it. In our example, we have used a certificate for encrypting the key. Because of the same reason, we are using the same certificate for opening the key and making it available for use. Subsequent to opening it and making it available for use, we can use the encryptkey function and store the encrypted values in the database, in the EncryptSecondCol column.

/* Update binary column with encrypted data created by certificate and key */
USE EncryptTest
GO
OPEN SYMMETRIC KEY TestTableKey DECRYPTION
BY CERTIFICATE EncryptTestCert
UPDATE TestTable
SET EncryptSecondCol = ENCRYPTBYKEY(KEY_GUID('TestTableKey'),SecondCol)
GO

We can drop the original SecondCol column, which we have now encrypted in the EncryptSecondCol column. If you do not want to drop the column, you can keep it for future comparison of the data when we decrypt the column.

/* DROP original column which was encrypted for protect the data */
USE EncryptTest
GO
ALTER TABLE TestTable
DROP COLUMN SecondCol
GO

We can run a SELECT query on our database and verify if our data in the table is well protected and hackers will not be able to make use of it even if they somehow manage to reach the data.

/* Check the content of the TestTable */
USE EncryptTest
GO
SELECT *
FROM TestTable
GO

Result of the previous SQL query

Authorized user can use the decryptbykey function to retrieve the original data from the encrypted column. If Symmetric key is not open for decryption, it has to be decrypted using the same certificate that was used to encrypt it. An important point to bear in mind here is that the original column and the decrypted column should have the same data types. If their data types differ, incorrect values could be reproduced. In our case, we have used a VARCHAR data type for SecondCol and EncryptSecondCol.

/* Decrypt the data of the SecondCol  */
USE EncryptTest
GO
OPEN SYMMETRIC KEY TestTableKey DECRYPTION
BY CERTIFICATE EncryptTestCert
SELECT CONVERT(VARCHAR(50),DECRYPTBYKEY(EncryptSecondCol)) AS DecryptSecondCol
FROM TestTable
GO

If you drop the database after the entire processing is complete, you do not have to worry about cleaning up the database. However, in real world on production servers, the database is not dropped. It is a good practice for developers to close the key after using it. If keys and certificates are used only once or their use is over, they can be dropped as well. Dropping a database will drop everything it contains – table, keys, certificates, all the data, to name a few.

/* Clean up database  */
USE EncryptTest
GO
CLOSE SYMMETRIC KEY TestTableKey
GO
DROP SYMMETRIC KEY TestTableKey
GO
DROP CERTIFICATE EncryptTestCert
GO
DROP MASTER KEY
GO
USE [master]
GO
DROP DATABASE [EncryptTest]
GO

Summary

Encryption is a very important security feature of SQL Server 2005. Long keys and asymmetric keys create unassailable, stronger encryption and stronger encryption uses lots of CPU to encrypt data. Stronger encryption is slower to process. When there is a huge amount of data to encrypt, it is suggested to encrypt it using a symmetric key. The same symmetric key can be encrypted further with an asymmetric key for additional protection, thereby adding the advantage of a stronger encryption. It is also recommended to compress data before encryption, as encrypted data cannot be compressed.

Click to Download Scripts

Reference : Pinal Dave (http://blog.SQLAuthority.com)

SQL SERVER – FIX : Error: 18486 Login failed for user ‘sa’ because the account is currently locked out. The system administrator can unlock it. – Unlock SA Login

Today, we will riffle through a very simple, yet common issue – How to unlock a locked “sa” login?

It is quite a common practice that SQL Server is hosted on a separate server than application server. In most cases, SQL Server ports or IP are exposed to the web, which makes them risk prone. For hackers, System Admin login “sa” is the preferred account which they use for hacking. In fact, a majority of hackers try to hack into SQL Server by attempting to login using “sa” account. Once hackers gain access to server using “sa” login, they get a complete control over the SQL Server.

Hackers apply Brute Force method to hack “sa” login. Brute Force method is an attempt to guess a password using every possible combination. Be it in your machine where SQL Server is hosted or in your domain, if you have a policy setting that disables any account after a certain number of unsuccessful attempts, it will also disable all your SQL Server logins including “sa” login. In this scenario, SQL Server will display the following error:

msg: 18486
Login failed for user ‘sa’ because the account is currently locked out. The system administrator can unlock it.

Fix/Solution/Workaround:

1) Disable the policy on your system or on your domain level. However, this may not be the most appropriate option as it will adversely affect your security protection level.

2) If this is a one-time issue, enable “sa” login WITH changing password of “sa” login.

ALTER LOGIN sa WITH PASSWORD = 'yourpass' UNLOCK ;
GO

3) If this is a one-time issue, enable “sa” login WITHOUT changing password of “sa” login.

ALTER LOGIN sa WITH CHECK_POLICY = OFF;
ALTER LOGIN sa WITH CHECK_POLICY = ON;
GO

4) If you are not using “sa” login, switch your authentication from mixed mode authentication to windows authentication to remove your “sa” login account.

5) BEST Practice: Create another user with systemadmin role having the same rights as “sa” login and let “sa” login get disabled. Use the newly created account as this will not be exposed on the Internet and for hackers it will be a tough nut to crack! They will find it difficult to guess the right password and moreover, they will not be able to do Brute Force attack over it.

I am eager to know if there are other options to solve this problem.

If you simply want to change the “sa” password, you can follow my previous article SQL SERVER – Change Password of SA Login Using Management Studio.

Please note that it is not mandatory to reboot SQL Server or restart SQL Server services after changing the password for “sa” login. If you are interested, go through all the comments and bring forth your opinion about this discussion.

Reference : Pinal Dave (http://blog.SQLAuthority.com)

SQL SERVER – Check if Current Login is Part of Server Role Member

I often work on consulting projects with umpteen clients from across the globe. The nature of the works I usually receive necessitates me to take on the role of a system admin. Now, this role is trailed by come common issues. This article revolves around one such concern.

Most of the time, I get login and DBA from my clients. However, sometimes I face login-related problems, primarily because my clients forget to confer admin rights on me. Anticipating this situation I perform a simple task, which saves my time and saves me from exasperation.  Whenever I receive a server login I run the following query to verify if  the client has assigned me the role of system admin or not.

SELECT IS_SRVROLEMEMBER('sysadmin');

If I have been assigned system admin role then the result will be 1 else it will be 0.

I have listed below valid roles that can be verified using the above function.

sysadmin
dbcreator
diskadmin
processadmin
serveradmin
setupadmin
securityadmin

If you pass an invalid role to the above function, it will return value NULL.

I hope my article is clear to all my readers. Please send me your feedback. I want my readers to come up with more issues that need to be tackled.

Reference : Pinal Dave (http://blog.sqlauthority.com)

SQL SERVER – Fix : Error : Msg 9803, Level 16. Invalid data for type “numeric” – Data Type Mapping for Oracle Publishers

My present article talks about an error that you will encounter when connecting to Oracle database using OPENQUERY.

The error that eventuates is as follows:

Msg 9803, Level 16, State 1, Line 1
Invalid data for type “numeric”.

Fix/Solution/Workaround:

As far as I can discern, the above error occurs due to numeric precision or numeric definition mismatch. The number field of SQL Server does not appropriately match with the number field of Oracle. In fact, apart from number field there are several other data types that do not match.

I am including the following sample query having NumberCol that is Integer field of SQL Server and it needs to be converted To_Char to match up. NumberCol can be matched to Numeric in Oracle as well.

SELECT CONVERT(INT, NumberCol) AS NumberCol
FROM OPENQUERY (YourConnectionMethod, 'SELECT TO_CHAR(NumberCol) AS NumberCol FROM RemoteTbl');

Do read the article of MSDN to have a sound knowledge about Data Type Mapping for Oracle Publishers.

Please drop a line to me and let me have your doubts and questions. Your suggestions are always welcome!

Reference : Pinal Dave (http://blog.sqlauthority.com)

SQL SERVER – 2008 – Activity Monitor is Empty – Fix Activity Monitor for All Users

This article is outcome of the technical discussion of activity monitor and its behavior with my friend and SQL Expert Tejas Shah. Tejas told me that he does not like to re-write content from MSDN but rather prefer to write real life scenarios, as that prepares him to become better SQL Expert. While discussing about Activity Monitor he informed that activity monitor throws an error when there is permissions issue. He has even blogged about how to give permissions to user to launch activity monitor on his blog . Tejas asked me to write on the same subject for SQL Server 2008. Here is the article covering the discussion I had with Tejas.

I have user called ‘ActivityUser’ when turned on Activity Monitor (while logging in with user ActivityUser’) it does not have show anything in Activity Monitor.

The issue here is permissions issue. If user ActivityUser is given all the necessary permission it will start showing up data in Activity Monitor. Activity Monitor is new tool in SQL Server which displays activity in five sections. 1) Overview, 2) Processes, 3) Resources Waits, 4) Data File I/O, 5) Recent Expensive Queries. It is one of the new and very useful tool introduced by SQL Server.

Activity Monitor captures all the information at server level. For the same reason we need to give “View Server State” permission to user name to view data of Activity Monitor.We can give permission either using T-SQL or using SSMS.

T-SQL to give permission to user to view Activity Monitor:

SSMS to give permission to user to view Activity Monitor:

Once permissions is given to user, it displays the data in Activity Monitor. Click on images to enlarge the images.

Additionally, note that if you are user belonging to sysadmin role, you can always see all the data in Activity Monitor without additional permissions.

Reference : Pinal Dave (http://blog.sqlauthority.com)

SQL SERVER – Find Current Location of Data and Log File of All the Database

As I am doing lots of experiments on my SQL Server test box, I sometime gets too many files in SQL Server data installation folder – the place where I have all the .mdf and .ldf files are stored. I often go to that folder and clean up all unnecessary files I have left there taking up my hard drive space. I run following query to find out which .mdf and .ldf files are used and delete all other files. If your SQL Server is up and running OS will not let you delete .mdf and .ldf files any way giving you error that file already in use. This list also helps sometime to do documentation of which files are in being used by which database.

SELECT name, physical_name AS current_file_location
FROM sys.master_files

Following is the output of files used by my SQL Server instance.

Reference : Pinal Dave (http://blog.SQLAuthority.com)