Monday, July 25, 2016

June 2016 Security Update Breaks Group Policy

So, this was a thing back in the middle of June and I missed it at the time. Looking back there are a few articles about it, but I just ran into this with a client (happily, before the update was applied) and I think it's worth raising awareness of.

In some security updates for Vista and later released on June 14, 2016 the process for Group Policy object (GPO) downloading has been changed. To make the download process more secure, the computer account is now responsible for downloading all GPOs. In the past, the user account could download the GPOs also.

The result of this change is that any computer that is downloading GPOs needs to have Read access to the GPO. The computer accounts do not need Apply permission, only Read is required.

By default the Authenticated Users group has Read and Apply permission for all GPOs. The Authenticated Users group includes all users and all computer accounts. So, if you haven't changed this, then you'll have no issues when these security updates are applied.

Even though you don't think you modified the permissions on your GPOs, you might have. If you use security filtering to control GPO application, that modifies the GPO permissions. When you remove Authenticated Users for security filtering, you also remove Read permission for Authenticated Users.

The quick fix is to add Authenticated Users or Domain Computers with Read permission to all of your GPOs. This doesn't modify which users or computers apply the GPOs, but it does give the computer accounts the necessary permissions to download the GPOs. Some of the links below show how to automate this process.

If you'd like to verify whether you have any GPOs without any permissions assigned to Authenticated Users, I've created a short script that looks for that and dumps the list of GPOs and their current permissions to a file.

 #Script Requires the GroupPolicy cmdlets  
 #A manual import of the GroupPolicy module is required for pre-Win2012  
 Import-Module GroupPolicy  
 #Define badGPO as an array or you get one big string  
 #that's hard to work with  
 $badGPO = @()  
 #Get a list of all Group Policy objects  
 $gpo = Get-GPO -All  
 #Find Group Policy objects that don't have permissions  
 #assigned to Authenticated Users  
 Foreach ($g in $gpo) {  
      Try { Get-GPPermissions -GUID $ -TargetName "Authenticated Users" -TargetType Group -ErrorAction Stop }  
      Catch { $BadGPO += $g.DisplayName }  
 Write-Host "List of GPOs without Authenticated Users permissions"  
 #Create File for Report  
 "Permissions Report of GPOs without Authenticated Users" | Out-File GPO-NoAuthUser.txt  
 Foreach ($b in $badGPO) {  
      "GPO: $b" | Out-File GPO-NoAuthUser.txt -NoClobber -Append  
      Get-GPPermissions -Name $b -All | Out-File GPO-NoAuthUser.txt -NoClobber -Append  
 Write-Host "Created GPO-NoAuthUser.txt with permissions report"  
 Write-Host "Use this report to verify that computer accounts"  
 Write-Host "have read access to the GPO for application after"  
 Write-Host "applying June 2016 Security Updates. For more info"  
 Write-Host "see:"  

Some links with more information:

    Friday, July 15, 2016

    A Reason To Use Preauthentication with Exchange

    Those that are really paranoid about security (and I don't mean that in a bad way) have always like the idea of pre-authentication in a reverse proxy for Exchange server. When you implement pre-authentication at the reverse proxy, only requests from authenticated users ever get to the Exchange server.

    This sounds great, but I think from a practical perspective, it doesn't buy you much. It also adds significant complexity. Even with vendor support, getting pre-authentication going always seems to be a hassle.

    Now, if you want to implement two factor authentication for Outlook on the web/OWA, then pre-authentication is a good point to do that. Most vendors have support for adding two factor authentication. For my customers, this is seldom a concern.

    Recently though I ran into an issue with account lockouts caused by repeated password attacks on OWA. In this case, the user account is locked and the user can't work. There is no simple solution for this without pre-authentication. Disabling OWA for the user doesn't prevent the failed authentication and the account lockouts. In complex environments with directory synchronization, changing a user logon name could break all sorts of things.

    Most pre-authentication solutions allow you to lock out accounts at the reverse proxy before you hit the limits for account lockout in Active Directory. The net effect is that OWA is locked but the user can continue to work internally.

    Now you need to think about whether that password attack is likely to happen.

    As an alternative to pre-authentication, you can also look at preventing automated sign in attempts. One way to do this is with a captcha on the OWA login page. Here are a few links on how to do this with free captcha software available from Google. This authentication is spoofable of the attacker builds their own login page to send the authentication requests, but they'd need to be pretty motivated.
    Another alternative that implements a captcha and also does much more is OWA Guard from Messageware. I haven't had a chance to install the trial yet, but it looks like a good product with the ability to do geo-blocking and OWA specific lockouts that will prevent denial of service for AD accounts when a password attack is being performed on OWA.

    Sunday, July 10, 2016

    Getting an Integer from Get-MailboxStatistics

    I was doing some analysis of an existing Exchange 2010 organization for a migration project and was wanting to calculate the average mailbox size. So, I used Get-MailboxStatistics to gather the information and export it to a csv file. Unfortunately, Get-MailboxStatistics gives you the mailbox size in this format:
    2.169 GB (2,329,318,152 bytes)
    When you're trying to use Excel to get an average, this isn't going to work. Instead you need to get that value as an integer. To do this, you need to create your own property for the object with an integer value.

    The TotalItemSize property from Get-MailboxStatistics is more complex than it appears. When you use Get-Member to look at the TotalItemSize property returned by Get-MailboxStatistics, it actually contains an IsUnlimited property and a Value property. The Value property has the size of the mailbox.

    If you do a Get-Member on the Value property, you can see that there are methods to convert the value to an integer. For example:
    (Get-MailboxStatistics Bob).TotalItemSize.Value.ToMB()
    To create a new property with the correct value as an integer, you use the Add-Member cmdlet. A complete 3 command example that gets all mailboxes and exports the size to csv is below:
    $mbx = Get-Mailbox -Resultsize Unlimited | Get-MailboxStatistics
    $mbx | Add-Member -MemberType ScriptProperty -Name MbxSizeInMB -Value {$this.TotalItemSize.Value.ToMB()}
    $mbx | Select-Object DisplayName,MbxSizeInMB | Export-Csv MailboxSize.csv
    Line 1 gets the mailboxes and their statistics. Line 2 adds the property. Line 3 exports the mailbox name and size to a csv file.

    Sunday, July 3, 2016

    Manual Reseed of Exchange Content Index

    I recently ran into an issue where a mailbox database content index was in a failed state after the Exchange 2010 server had a power outage. Not too big of a deal since it was in a remote site that was hosting only passive databases for disaster recovery.

    Normally for a passive copy, you run the following command and all is good:
    Update-MailboxDatabaseCopy -Identity DBName\ServerName -CatalogOnly

    However I got this error:
    A source-side operation failed. Error An error occurred while performing the seed operation. Error: An error occurred while updating the search catalog files from server 'XXXXXXX'. Error: A transient exception from Exchange Search was encountered. Error: Catalog was not paused for indexing on database dbGUID.

    There were only a couple of search hits for this specific error about pausing indexing and they really only indicated it was an internal process as part of the catalog reseed. There were no useful errors in the event log.

    I tried a few different variations with no success:
    • initiate from passive
    • initiate from active
    • suspend passive copy before updating catalog
    • remove catalog files on passive before starting
    • force AD replication in case replication lag was causing an issue
    Finally, what worked was the following:
    1. Stopped the Microsoft Exchange Search Service on both the active and passive copies.
    2. Deleted the catalog folder on the passive.
    3. Manually copied the catalog folder from the active to the passive.
    4. Started the Microsoft Exchange Search Service on the active.
    5. Started the Microsoft Exchange Search Service on the passive.
    After following this process, the content index on the passive copy is good.

    Tuesday, June 7, 2016

    Error Importing Hyper-V Virtual Machine

    I regularly get Hyper-V VMs for use developing course materials. Our environment is pretty standardized, so, there are seldom any issues. However, today when running our import script, I got the following error for several of the VMs in a new course:
    Import-VM : Unable to import virtual machine due to configuration errors. Please use Compare-VM to repair the virtual machine.
    So, I ran Compare-VM and placed the output in a variable for easier viewing:
    $report=Compare-VM -Path .\GUID.xml
    The mystery was solved when I looked at the incompatibilities property in the output:

    The incompatibilities showed that a a virtual network was missing:
    Message: Could not find Ethernet switch 'Cluster Network'.
    MessageID: 33012
    Source: Microsoft.HyperV.PowerShell.VMNetworkAdapter

    When I took a look at my virtual networks, I saw that the script I ran to create the virtual networks created Cluster_Network instead of "Cluster Network". After I renamed the network, then all was good.

    Tuesday, May 31, 2016

    Error Using PowerShell Direct

    If you've not heard of it, PowerShell Direct is a nifty new way that you can connect with virtual machines on a Windows 10 or Windows Server 2016 Hyper-V host. PowerShell Direct lets you enter PowerShell remoting sessions or invoke commands on a VM from the Hyper-V without requiring network connectivity.

    To enter a PowerShell remoting session:
    Enter-PSSession -VMName NameOfVM
    I've played with this a bit before and it's pretty cool. I've never had any issues with it. However, today when working with some VMs provided for a course I'm working on I got longer version of this error when trying to connect:
    Enter-PSSession : An error has occurred which Windows PowerShell cannot handle. A remote session might have ended.
    + FullyQualifiedErrorId :  CreateRemoteRunspaceForVMFailed,Microsoft.PowerShell.Commands.EnterPSSessionCommand
    This same error message is provided if your authentication credentials fail. So, I reset the local administrator password on the VM, but no fix.

    This was a generation 1 VM given to me. So, my first throught is that maybe its the generation of VM. Nope. I tested with other generation 1 and generation 2 VMs and all worked fine.

    What it finally ended up being was the version of VM. The VM was built on a Windows Server 2012 R2 host. So, it was version 5 rather than version 7.1 for the VMs that I had created.

    To view the version:
    Get-VM | FT Name,Version
    To update the version:
    Update-VMVersion NameOfVM
    After updating the VM to from version 5 to version 7.1 PowerShell Direct worked just like it is supposed to. This is worth noting because none of the articles about PowerShell Direct mention the version as a required.

    Tuesday, April 5, 2016

    Dell, Broadcom and Virtual Machine Queues

    We work with Dell servers and they come with Broadcom network cards. In general they work well, but they have an issue when being used as Hyper-V hosts. Virtual machine queues which in theory improve performance end up bogging down networking.

    Symptoms we've seen:
    • Slow file copying to/from VMs over the network.
    • Dropped network connection for entire host that is fixed by reboot.
    In both cases the fix is to disable virtual machine queues (VMQ). We had been doing it in the properties of the physical network on the Hyper-V host. However, we were recently having issues with a host and that option wasn't in the interface provided by the driver.

    Some blogs were referring to using registry edits to disable it. However, a faster and easier way in Windows Server 2012 R2 (maybe also Windows Server 2012, but I haven't verified) is by using Windows PowerShell.

    To view the VMQ status of your network adapters:

    To disable VMQ for all adapters:
    Get-NetAdapterVmq | Disable-NetAdapterVmq