Sunday, March 28, 2010
What if you forget the password of Database Master Key (DMK)
Database Master Key (DMK)
USE TestDataBase GO CREATE MASTER KEY ENCRYPTION BY PASSWORD = 'SQLReality_1'
USE TestDataBase GO OPEN MASTER KEY DECRYPTION BY PASSWORD = 'SQLReality_1'
USE TestDataBase GO ALTER MASTER KEY ADD ENCRYPTION BY SERVICE MASTER KEY
Service Master Key (SMK)
Difference between Cell- level Encryption and Transparent Data Encryption (TDE)
| Cell- level Encryption | Transparent Data Encryption (TDE) |
| Granular control over which data is encrypted | Encrypts the entire database |
| User‐aware encryption can control access on a need‐to‐know basis | Encryption is not user‐aware; data is open to all users who have permission to access the database |
| Requires analysis to find sensitive data | No analysis required because entire database is encrypted |
| Affect table structure because encrypted data can only be stored in varbinary data type columns. | No change is required in table structure. |
| Database applications need to be modified to use specific functions to encrypt and decrypt data. | No database application change needed. |
| Indexes, primary keys, and foreign keys cannot be encrypted. | No impact on indexing, primary keys, or foreign keys |
| Potential impact on performance because indexes on encrypted columns are not used while searching a value. | Small impact on performance (up to 5%) |
Facts about SQL Server Data Encryption & Decryption
· Password (ENCRYPTBYPASSPHRASE function)· Hashing (HASHBYTES function)· Symmetric Key· Asymmetric key· Certificate· TDE
· Encrypt high-value and sensitive data only.· Use symmetric keys to encrypt data, and asymmetric keys or certificates to protect the symmetric keys.· Password-protect keys and remove master key encryption for the most secure configuration.· Always back up the service master key, database master keys, and certificates by using the key-specific DDL statements.· Always back up your database to back up your symmetric and asymmetric keys.
15. Data can be protected if we take control over automatic key management. This can be done by following three methods:
1. Granting authorization on key to authorized users only.16. Password can be used to encrypt table data and any level of key up to DMK.
2. Encrypting the data or keys with secure password.
3. Extensible Key Management (EKM).
Using OUTPUT clause for auditing
CREATE TABLE [dbo].[MyTable1](
[Col1] [int] NULL,
[Col2] [varchar](10) NULL
) ON [PRIMARY]
CREATE TABLE [dbo].[MyTable1_Audit](
[Col1_Old] [int] NULL,
[Col1_New] [int] NULL,
[Col2_Old] [varchar](10) NULL,
[Col2_New] [varchar](10) NULL,
[Action] varchar(50) NULL
) ON [PRIMARY]
GOINSERT INTO MyTable1OUTPUT NULL, inserted.Col1, NULL, inserted.Col2, 'INSERT'
INTO [MyTable1_Audit]VALUES (1,'WOW')
SELECT * FROM MyTable1
SELECT * FROM [MyTable1_Audit]
UPDATE MyTable1 SET Col1 = 10 , Col2 = 'Great'
OUTPUT deleted.Col1, inserted.Col1, deleted.Col2, inserted.Col2, 'UPDATE' INTO [MyTable1_Audit]
WHERE Col1 = 1SELECT * FROM MyTable1
SELECT * FROM [MyTable1_Audit]
Saturday, March 6, 2010
Get free space in Data and Log files in all databases
To get total size and free space (unallocated space) in transaction log files of all databases we have a very handy command:
DBCC SQLPERF(LOGSPACE)
But there is no such command to get the free space in data files of all databases. Even in current database, data file size and free space can be checked by sp_spaceused.
But if you have many databases than connecting to each database and executing the sp_spaceused command is troublesome.
To simplify this task I have written following query. This provides you all information about file size and free space in all databases.
CREATE TABLE #temp (DatabaseName VARCHAR(100), DataFileSize_MB INT,
Free_Space_in_Data_Files_MB INT, LogFileSize_MB INT)
INSERT INTO #temp EXEC sp_msforeachdb 'SELECT ''?'' as DatabaseName, DBSize.DataSize,
case when DBSize.DataSize - allocated.ReservedPages > 0 then
DBSize.DataSize - allocated.ReservedPages else 0 end as FreeSpace_MB, DBSize.LogSize
FROM (SELECT (sum(convert(bigint,case when status & 64 = 0 then
size else 0 end))* 8192 / 1048576) DataSize,
(sum(convert(bigint,case when status & 64 <> 0 then
size else 0 end))* 8192 / 1048576) LogSize
FROM [?].dbo.sysfiles) DBSize,
(SELECT (sum(total_pages)* 8192 / 1048576) AS ReservedPages
FROM [?].sys.allocation_units) allocated'
SELECT instance_name AS 'DatabaseName', tt.DataFileSize_MB, tt.Free_Space_in_Data_Files_MB,
tt.LogFileSize_MB, (tt.LogFileSize_MB -(MAX(cntr_value)/1024)) AS 'Free_Space_in_Log_Files_MB'
FROM #temp tt inner join sysperfinfo on tt.DatabaseName = sysperfinfo.instance_name
WHERE counter_name ='Log File(s) Used Size (KB)' AND instance_name != '_total'
GROUP BY instance_name , tt.DataFileSize_MB, tt.Free_Space_in_Data_Files_MB, tt.LogFileSize_MB
ORDER BY tt.Free_Space_in_Data_Files_MB + (tt.LogFileSize_MB -(MAX(cntr_value)/1024)) DESC
--drop table #temp
Following is the screenshot of its output
Friday, March 5, 2010
Best practices to write complex JOINs on multiple tables
SELECT tableA.colX, tableB.colY FROM tableA, tableB WHERE tableA.col1= tableB.col1
--ANSI standard:
SELECT tableA.colX, tableB.colY FROM tableA INNER JOIN tableB ON tableA.col1= tableB.col1
In earlier version of SQL Server, Transact-SQL standard had *= and =* operator for LEFT and RIGHT OUTER joins. But now these operators have been deprecated. So always use the ANSI standard for unanimity and clarity. Find the easier to understand query out of following two same queries:
--Query 1: SELECT tableA.colX, tableB.colY, tableC.colZ FROM tableA, tableB LEFT OUTER JOIN tableC ON tableA.col2 = tableC.col2 WHERE tableA.col1= tableB.col1 --Query 2: SELECT tableA.colX, tableB.colY, tableC.colZ FROM tableA INNER JOIN tableB ON tableA.col1= tableB.col1 LEFT OUTER JOIN tableC ON tableA.col2 = tableC.col2
--Query 1: Using left outer join SELECT tableA.colX, tableB.colY, tableC.colZ FROM tableA LEFT OUTER JOIN tableB ON tableA.col1= tableB.col1 LEFT OUTER JOIN tableC ON tableB.col2 = tableC.col2 --Query 2: Using left and right outer join<?xml:namespace prefix = o />
SELECT tableA.colX, tableB.colY, tableC.colZ FROM tableB RIGHT OUTER JOIN tableA ON tableA.col1= tableB.col1 LEFT OUTER JOIN tableC ON tableB.col2 = tableC.col2 --Query 3: Using right outer join<?xml:namespace prefix = o />
SELECT tableA.colX, tableB.colY, tableC.colZ FROM tableC RIGHT OUTER JOIN tableB ON tableB.col2 = tableC.col2 RIGHT OUTER JOIN tableA ON tableA.col1= tableB.col1
--Query 1: SELECT tableA.colX, tableB.colY, tableC.colZ FROM tableA LEFT OUTER JOIN tableB ON tableA.col1= tableB.col1 INNER JOIN tableC ON tableA.col2 = tableC.col2 --Query 2: SELECT tableA.colX, tableB.colY, tableC.colZ FROM tableA INNER JOIN tableC ON tableA.col2 = tableC.col2 LEFT OUTER JOIN tableB ON tableA.col1= tableB.col1
--Query 1: SELECT tableA.colX, tableB.colY, tableC.colZ FROM tableA INNER JOIN tableB LEFT OUTER JOIN tableC ON tableA.col1= tableB.col1 ON tableA.col2 = tableC.col2 --Query 2: SELECT tableA.colX, tableB.colY, tableC.colZ FROM tableA INNER JOIN tableB ON tableA.col1= tableB.col1 LEFT OUTER JOIN tableC ON tableA.col2 = tableC.col2
If I am to write a query to know the Salesperson, product and the total sold quantity of that product, I have to identify the relation between Salesperson, Product and there sold quantity. Following is the screenshot of database diagram of required tables in AdventureWorks database:
Now when we know the relation, we can write a query to join these tables:
SELECT Emp.LoginID, Prod.Name, SUM (SOD.OrderQty) AS TotalQuantitySold
FROM HumanResources.Employee Emp LEFT OUTER JOIN Sales.SalesOrderHeader SOH
ON Emp.EmployeeID = SOH.SalesPersonIDINNER JOIN Sales.SalesOrderDetail SOD
ON SOH.SalesOrderID = SOD.SalesOrderIDINNER JOIN Production.Product Prod
ON SOD.ProductID = Prod.ProductIDGROUP BY Emp.LoginID, Prod.Name
ORDER BY Emp.LoginID, Prod.Name
The screenshot of query result is following:
Thursday, March 4, 2010
The purpose of Transaction Log
We all know that all DDL and DML changes are logged in Transaction log. But when I ask a question that what is the primary purpose of Transaction log in SQL Server then the most common answers that I get are following:
· For Transaction log backup.
· For Transactional Replication, Log shipping or Mirroring.
As we know these two features can not be implemented in SIMPLE recovery model but transaction log is still maintained in this recovery model. This indicates that even these features use transaction log but these are not the prime purpose of Transaction log.
The primary purpose is Rollback & Recovery.
We all know that reversing the changes is called Rollback in SQL Server. This is the process to keep the database consistent and integrated. Transaction is the unit of action and either all statements of a transactions complete successfully or none.
So what is Recovery? Recovery includes two steps: Redo and Undo. In Redo all changes of completed transactions, after the last checkpoint are updated in data pages. In Undo process all changes of uncompleted transactions up to last checkpoint are rolled back.
Recovery is performed on a database in two cases:
1. While starting the database when SQL Server starts
2. While restoring the database.
The recovery process while restoring a database can be paused for later time by using NORECOVERY option in RESTORE DATABASE or RESTORE LOG command.
Wednesday, February 24, 2010
Find who dropped, created or altered the table, SP or any object
If any database object is dropped, created or altered accidently, sometimes you may need information about:
Who dropped, created or altered the object? When culprit not accept the mistake.
When the object was dropped, created or altered? Especially drop time of table is required for point in time recovery from database backups.
These changes are not recorded in SQL Server Error Log but are recorded in default trace. If you have not disabled the default trace and started looking into the issue soon after change occurred you could get this information. The path of trace file is “C:\Program Files\Microsoft SQL Server\MSSQL10.SQLEXPRESS\MSSQL\Log\”. The folder MSSQL10.SQLEXPRESS may be different according to your SQL Server instance name. Here you would find 5 profiler trace files named as “log_1”, “Log_2”, etc. The file number is increased by time (in following example I used “log_16.trc”). Find the file that is most recently modified and use the below t-sql query to open that file:
SELECT ObjectName, DatabaseName, StartTime, NTDomainName, HostName, NTUserName, LoginName
FROM fn_trace_gettable('C:\Program Files\Microsoft SQL Server\MSSQL10.SQLEXPRESS\MSSQL\Log\log_16.trc', default)
WHERE objectname is not null
You would get the output as shown below:
Tuesday, February 23, 2010
Some unknown facts about Update (U) lock
You may have read and found the following statement type – lock type relation easy to understand and logical too:
SELECT – Shared (S)
UPDATE – Update (U)
INSERT & DELETE – Exclusive (X)
I was clear about the shared (S) and exclusive (X) but not about update (U) lock. Some of my questions about update lock were following:
Is it used only while updating a record or while deleting the record too?
As update (U) lock is converted to exclusive (X) lock then how long update (U) lock persist on a row?
To get the answer of these questions I did some analysis and I concluded that work in following points:
1. Update lock is used not only with UPDATE statements but also with DELETE statements.
2. Update lock is used only if a transaction wants to update or delete a record that is already locked (S) by another transaction. Otherwise direct X lock is used. In READ UNCOMMITTED isolation level, no shared (S) lock is used so update (U) lock can never be found in GRANT status (in sp_lock output). We can only find it in WAIT status.
3. Update lock is “Waiting exclusive lock”. So we can never find an exclusive (X) lock in WAIT status (in sp_lock output) because in that case update (U) lock is used.
Please let me know your analysis and review.
Sunday, February 21, 2010
The use of Synonyms
CREATE SYNONYM Employee FOR TestServer.Sales.dbo.Employee
DROP SYNONYM EmployeeGOCREATE SYNONYM Employee FOR UATServer.Sales.dbo.Employee
SELECT TOP 5 Emp.EmpName, Mng.EmpName
FROM TestServer.Sales.dbo.Employee Emp INNER JOIN
(SELECT EmpId, EmpName FROM TestServer.Sales.dbo.Employee) Mng
ON Emp.MngID = Mng.EmpIDSELECT TOP 5 Emp.EmpName, Mng.EmpName
FROM Employee Emp INNER JOIN
(SELECT EmpId, EmpName FROM Employee) Mng
ON Emp.MngID = Mng.EmpIDHere you can find the difference in the easiness while writing and reading the statement.
Synonyms in SQL Server
1. Synonym can be used to refer an object in current server or remote server.2. Synonym can be created for following types of objects:a. Tables: User-defined that include local and global temporary tables alsob. Stored Procedures: SQL, CLR and extended, all types of stored proceduresc. Functions: SQL and CLR both types of functionsd. Views3. A synonym can not be the base object for another synonym.4. Dropping a synonym does not affect the base object.5. Synonyms are not schema bound to base object and remain deferred until run-time. In other words the base object’s existence, validity, and permissions are checked only at run time.6. Syntax to create a synonym is following:CREATE SYNONYM [schema_Name.]Synonym_Name FOR base_Object7. Syntax to drop a synonym is following:DROP SYNONYM [schema_Name.]Synonym_Name8. Synonym can be used in DML statements. Let us test with following code:CREATE TABLE dbo.tblTest (Col1 INT, Col2 INT)GOINSERT INTO dbo.tblTest VALUES(1,10)GOCREATE SYNONYM syn_Table FOR dbo.tblTestGOSELECT * FROM syn_Table -- retunrs 1 recordGOINSERT INTO syn_Table VALUES (1,1)SELECT * FROM syn_Table -- returns 2 records
9. Synonym can not be used in DDL statements. If we execute the below statements:
ALTER TABLE syn_Table ADD Col3 INT
Following error occur:Msg 4909, Level 16, State 1, Line 1Cannot alter 'syn_Table' because it is not a table.
8. Synonyms are like view in permission checking and ownership chaining. For example if an user TestUser does not have execute permission on a stored procedure dbo.proc_TestSynonym, but have execute permission on a synonym of this stored procedure dbo.syn_Proc then user can execute dbo.syn_Proc successfully. Test this using below script:
CREATE PROCEDURE dbo.proc_TestSynonym
ASSELECT * FROM dbo.tblTest --change the table name
GOCREATE LOGIN TestUser WITH PASSWORD = 'TestUser', CHECK_POLICY = OFF
GOUSE Test --DB nameGOCREATE USER TestUser
GODENY EXECUTE ON dbo.proc_TestSynonym TO TestUser
GO CREATE SYNONYM dbo.syn_Proc FOR dbo.proc_TestSynonym
GO GRANT EXECUTE ON dbo.syn_Proc TO TestUser
GO EXEC AS USER = 'TestUser'
GO PRINT '1'
EXEC dbo.proc_TestSynonym --fail with permission denied error
GO PRINT '2'
EXEC dbo.syn_Proc --successfully executes GO REVERT
GO DROP SYNONYM dbo.syn_ProcGODROP PROCEDURE dbo.proc_TestSynonym
GODROP USER TestUser
GODROP LOGIN TestUserSunday, January 31, 2010
Life time (age) of Shared, Exclusive and Update lock
SQL Server uses locks to maintain consistency while multiple transactions execute simultaneously. We know the various types of locks at various levels. But what is the life time or age of a lock? Let me answer it one by one for three types of locks:
Shared Locks (S): Age of shared lock is decided by the isolation level of session. Following table describes the isolation level and age of Shared lock:
| Isolation Level | Age of Shared Lock |
| READ UNCOMMITTED | No lock held |
| READ COMMITTED | Execution time of SELECT statement |
| READ COMMITTED WITH SNAPSHOT | No lock held |
| REPEATABLE READ | Execution time of outermost user defined Transaction |
| SNAPSHOT | No lock held |
| SERIALIZABLE | Execution time of outermost user defined Transaction |
Exclusive Locks (X): Execution time of outermost user defined Transaction.
Update Locks (U): Execution time of outermost user defined Transaction.
Yes, age of exclusive locks and update locks is always the length of outermost user defined transaction. There is no impact of isolation levels on the life time of these locks.
This behavior is same for row or key, page and table levels. Age of “intent lock” is same as the age of their base lock like Intent Shared (IS) lock’s age is same to that of its relevant Shared (S) lock.