In the past, I have written a few blogs where I explained a few issues about the failure of listener creation Failure in a cluster. In this blog, we would see another cause of listener creation failure.
Whenever I get issues while creating a listener in SQL Server, I look at the failover cluster manager to figure out the resource which is failing and then I generate a cluster log. If you are not sure how to generate cluster logs, read my earlier blog on the same topic. SQL SERVER – Steps to Generate Windows Cluster Log?
Here is the setup of my lab. This would help you in co-relating the cluster log from your environment.
- Listener Name – ProdListener
- Windows Cluster Name – WinCluster
As I mentioned my listener was failing, which is a client access point in the cluster manager. In the below lines from the cluster log, I have removed the timestamp to avoid clutter. Here is the trimmed version of the cluster log from the node where the network name resource was not coming online.
INFO [RES] Network Name : AccountAD: Initializing Name: ProdListener, NetbiosName: ProdListener, Type: Singleton, Created: false
INFO [RES] Network Name : AccountAD: PopulateNetnameADState – DCFlags 1073745937 LdapReferralAllowed 0
INFO [RES] Network Name: [NNLIB] FindSuitableDCNew – objectName ProdListener, username – WinCluster$, firstChoiceDCName – \\domain.com
INFO [RES] Network Name: [NNLIB] Found first choice DC that has the object ProdListener DCName – \\domain.com.
INFO [RES] Network Name: [NNLIB] GetOUForVCOCreation – OU name of CNO is OU=IN,DC=Domain,DC=Com
INFO [RES] Network Name: [NNLIB] GetOUForVCOCreation – CNO SID is S-1-5-21-2987268983-2468375096-37832846-293957
INFO [RES] Network Name : AccountAD: Object ProdListener already exists, using DC: \\domain.com
INFO [RES] Network Name: [NNLIB] Trying to enable disabled object ProdListener on DC: \\domain.com (Flags 0000000000001022)
WARN [RES] Network Name : AccountAD: Enabling object ProdListener failed with error 5
In above cluster log, we can see INFO, WARN, ERR which is similar to what we see in windows event logs, Warning, Information and Error. The most relevant error is “Network Name <ProdListener>: AccountAD: Enabling object ProdListener failed with error 5”. This essentially means that Cluster is trying to enable the object and failing with access denied error. They key part here is that this operation is done by “username – WinCluster$”. This is not a domain user but a cluster computer object, also known as CNO.
“Access is denied” error can be caused due to many other reasons which can cause network name or client access point resource in cluster creation to fail. Based on my experience one of the most common causes would be where the Domain Administrator does not allow the CNO “Read All Properties” and “Create Computer Objects” permissions. You might see “Access is denied” in the event log.
WORKAROUND/SOLUTION – Listener Creation
Based on the cluster log snippet, it was clear that a domain admin needs to provide access on VCO to CNO. VCO is a virtual computer object which is a listener in Always On scenario. CNO is a cluster name object which is a computer object for Windows Cluster network name in active directory.
- Open the Active Directory Users and Computers Snap-in (dsa.msc) on a domain controller.
- In Menu > View -> check Advanced Features. (Otherwise, we would not see option explained in next steps)
- Right-click the OU/Container where we want the VCO (ProdListener in above example) to be created and click “New” -> “Computer”
- Provide a name for the object (This will be Listener Name for AG) and click “OK”.
- Right-click on the VCO which we just created and select “Properties”. Click the Security tab and then click “Add”:
- Enter the CNO (Make sure to select “Computers” option in the “Object Types” window) and click “OK”. The CNO is a Cluster Name Object. This is the name of the Windows Cluster name NOT listener or FCI name.
- Give CNO “Full Control” over the VCO.
After this, we should be able to bring VCO online in the cluster manager. Listener creation also worked and it was a happy ending.
Reference: Pinal Dave (https://blog.sqlauthority.com)
Very helpful thank you for posting
I’ve found an issue when attempting this on our own new setup. The difference is we have multi-site AG. We have two sites main and DR. Primary SQL server is in the main, backup node is in the DR. When I created the cluster initially, the CNO was automatically created in our DR site OU, maybe because the server names from the DR site are alphabetically higher. No matter which order I’d add the servers in the cluster creation, the DR site server always came first, and the CNO is always created there. Cluster and AG create without issue.
The issue comes when attempting to create a listener. SQL attempts to create it in the same OU as the primary server (main site). When I give the CNO “create object” rights over this OU, it still fails (19471). The strange thing is that the icon for the CNO in the permissions dialog looks like a computer. When I move the CNO to the same OU as the primary SQL node in the main site and attempt the permissions again, the icon for the CNO in the permissions dialog looks like a user. Only then does the listener create without issue.
I’ve attempted this multiple times, and have been able to produce it each time.
Servers are all running Win2019, SQL2019. Domain is running in 2012 compatibility.
I’ve discovered, the hard way, that you must assign the Create Computer Object rights directly to the CNO, you cannot use a security group that contains the CNO.