Voicemail not Working Following a Disaster

Our headquarters suffered an outage, forcing us to implement our full disaster recovery scenario. Fortunately, it only lasted about 2 days before we could fail back.

Being responsible for our Lync environment, I was pretty pleased with how well Lync came back following a full and immediate power outage at our headquarters location. The only issue I came across that evening was that sharing via a conference was not working for edge-connected users. I found the fix to that via Elan Shudnow’s blog.

We have a few SBA’s scattered throughout our network. The day after our failback from DR I got an e-mail from the client care director at one of our offices asking me if I was aware that voicemail was not working at their location. I was not aware of this as it was working in my tests and no one else had reported any issues. So I started troubleshooting this issue.

I looked on one of our Exchange UM servers looking for errors in the event log. I found nothing relevant.

Then I looked in the Application section of Event Viewer on the SBA looking for errors. I found the following 2 errors, one right after the other:

Log Name:      Lync Server
Source:        LS Inbound Routing
Date:          11/1/2011 9:35:47 AM
Event ID:      45030
Task Category: (1037)
Level:         Warning
Keywords:      Classic
User:          N/A
Computer:      SBA0123.contoso.com
Description:
A missed call notification could not be generated for user sip:first.last@contoso.com because the assigned dial plan <dial plan name> is not known. Problems with this user’s dial plan will not be reported again for another hour.
Cause: The dial plan is no longer defined, or permission to access Active Directory objects may not be granted to this process.
Resolution:
Assign the user to a new dial plan, or adjust the permissions on Active Directory.

Log Name:      Lync Server
Source:        LS Exchange Unified Messaging Routing
Date:          11/1/2011 9:35:47 AM
Event ID:      44008
Task Category: (1040)
Level:         Error
Keywords:      Classic
User:          N/A
Computer:      SBA0123.contoso.com
Description:
Dial Plan Unknown

Dialplan [dial-plan-name] is not recognized by routing application
Cause: Dial plan does not exist, or Microsoft Lync Server 2010 does not have permission to read the relevant Active Directory objects.
Resolution:
If the dialplan is valid, then run exchucutil.ps1 in appropriate Exchange forest to give permission to Microsoft Lync Server 2010. If the dialplan is not valid, then clean up proxyAddresses attribute for the affected users.

This didn’t necessarily mean much to me. I did as they recommended and re-ran the exchucutil.ps1 on an Exchange server via the Exchange Powershell. It came back and essentially said “no changes made” meaning we were set up correctly.

I then fired up Lync powershell to determine what the proxyAddress mentioned might be. so I ran get-csaduser and toward the bottom found the ProxyAddresses entry. It looked something like this:

 {eum:1234;phone-context=, EUM:Ma… (cut off due to powershell screen width)

Armed with this clue, I next looked at user’s Exchange account and specifically the E-Mail Addresses tab. In there I looked at the EUM addresses entry. Both had the dial plan mentioned in the error above. I poked around and realized that the references all this time is the dial plan on Exchange (not on Lync). It is also the ONLY dial plan we have defined on Exchange. So if there were a problem with the dial plan, it would probably impact everybody.

So I scheduled a time with the users in that office to restart the 3 Lync services on their SBA. It only took about 15 seconds but when it came back, I soon got reports from 4 different people confirming that Voice Mail is now working correctly.

So the take home here is that during a failback from a disaster, go ahead and restart the Lync services on the various SBA’s as a final step before you call it a night.