When I got a call from my client, they were into the situation where availability group was not coming online. After an investigation, we concluded that their windows cluster was having issues and it was unable to start. The solution in such disaster is to start windows cluster in force quorum mode.
Once we started to cluster it in ForceQuorum mode (also called as fix quorum) we tried deleting the AG and received below error.
Failed to destroy the Windows Server Failover Clustering group corresponding to availability group ‘AG’. The operation encountered SQL Server error 41000 and has been terminated. Refer to the SQL Server error log for details about this SQL Server error and corrective actions.
An error occurred while removing availability group ‘AG’. The DROP AVAILABILITY GROUP command removed the availability group configuration from the local metadata. However, the attempt to remove this configuration from the Windows Server Failover Clustering (WSFC) cluster failed because the Always On Availability Groups manager is not online (SQL Server error: 41081). To remove the availability group configuration from the WSFC cluster, re-enter the command. (Microsoft SQL Server, Error: 41081)
In above error message, we see another error number: 41000 for which the text is as follows. “Failed to obtain the local Windows Server Failover Clustering (WSFC) handle (Error code %d). If this is a WSFC availability group, the WSFC service may not be running or may not be accessible in its current state. Otherwise, contact your primary support provider. For information about this error code, see “System Error Codes” in the Windows Development documentation.”
WHAT YOU SHOULD DO?
Well, no action needed. The message raised is not a dangerous error message. It clearly mentions that availability group information has been removed from SQL Server, but it might still exist in the failover cluster. So, you can click OK on message and check availability group in SSMS and it should be gone from “Always On High Availability” > “Availability Group”. At this point, the database would be in restoring state and you need to bring them online using below
ALTER DATABASE <DBName> SET ONLINE
RESTORE DATABASE <DBName> WITH RECOVERY
Once things are back to normal, you need to reconfigure availability again.
Reference: Pinal Dave (https://blog.sqlauthority.com)