Saturday, January 24, 2009

SQL Server has encountered occurrence(s) of cachestore flush for the 'Bound Trees' 'SQL Plans' 'Object Plans' cachestore (part of plan cache)

Today one of server had HBA adaptor issue. Redundant link to SAN Drive had some problem, because of which LDF was not accessible to SQL Server for sometime and because of this SQL Server got hung. After reseating HBA adaptor this issue got resolved. Below are some error messages we have noticed and solution for them.

Error Message:
SQL Server has encountered 3 occurrence(s) of cachestore flush for the 'Bound Trees' cachestore (part of plan cache) due to some database maintenance or reconfigure operations.
SQL Server has encountered 3 occurrence(s) of cachestore flush for the 'SQL Plans' cachestore (part of plan cache) due to some database maintenance or reconfigure operations.
SQL Server has encountered 3 occurrence(s) of cachestore flush for the 'Object Plans' cachestore (part of plan cache) due to some database maintenance or reconfigure operations.


Root Cause:
1) The cache for a DB gets flushed when the DB is restored, taken offline or detached. Also when some ALTER Database statements are run.The cache for the entire server gets flushed when some configuration changes are made and then there's the explicit DBCC statements to empty the cache.You should try and identify what happend to cause that from error log. Most of the operations that will flush a cache require fairly high permissions.Cache flushed aren't nice on busy servers, because SQL then has to repopulate all the caches. It causes slowdowns.
Check below link for complete causes for Cache Flush
http://blogs.msdn.com/sqlprogrammability/archive/2007/01/17/10-0-plan-cache-flush.aspx
2) Another root cause is "AUTO_CLOSE" option enabled for production databases which will cause this message to be printed in error log (Starting from SQL 2005 SP2). As per thumbrule, Never ever enable this AUTO_CLOSE option in production DB's. By default this option is turned ON in MSDE and Express editions.

Error Message related to LDF not accessible when there was HBA Adaptor issue:
The operating system returned error 2(The system cannot find the file specified.) to SQL Server during a read at offset 0000000000000000 in file 'E:\Abc\SAPDB\UserDB3\DBdat.mdf'. Additional messages in the SQL Server error log and system event log may provide more detail. This is a severe system-level error condition that threatens database integrity and must be corrected immediately. Complete a full database consistency check (DBCC CHECKDB). This error can be caused by many factors; for more information, see SQL Server Books Online.

During undoing of a logged operation in database 'DB', an error occurred at log record ID (25015:9074:1). Typically, the specific failure is logged previously as an error in the Windows Event Log service. Restore the database or file from a backup, or repair the database.

LogWriter: Operating system error 2(The system cannot find the file specified.) encountered.

Root Cause:
This happens purely due to hardware issue (Storage or Device Driver). In our case it was SAN Adaptor issue.

Solution:
1) Fix the hardware issue and restore database from the recent backup. But there is quite a possibility that the backup doesn’t exist or may be incomplete. So what to do next?
2) Use MDF Recovery Softwares to recover maximum possible data.
There are third party applications which are designed to target the same. However for resisting the future problems, have the latest hotfix released by Microsoft and for the present situation, you can use the SQL repair applications.
These MDF recovery software have the extensive power to scan the database systematically and retrieve the information. The less technically saavy user can end the job of MDF repair, with much ease and effectiveness.
You can blindly trust on Stellar Phoenix SQL Database Recovery for having the safe and valuable SQL recovery. This application uses the most effective scanning algorithms to scan, repair and restore the database. This SQL repair software is found to be compatible with SQL Server 2000 and works for MDF recovery in case of corruption. The safe MDF repair is guaranteed due to its read-only nature.