Master & Cmd-R

Restoring Outlook Categories

Similar to Autocomplete settings, Outlook Categories are saved in a Stream_CategoryList_1_guid.dat file. The GUID is random, and looks something like this:


In order to restore Outlook Categories when creating a new profile, you can use the same approach required for restoring Autocomplete files. Basically, rename the old categories .dat file to match the DAT file associated with your new profile.

Here’s what my categories looked like before creating a new profile:


And here is the generic categories that Outlook comes with:


Not only am I missing my custom categories which prevents me from setting the categories I want on all my new emails, I’m also missing the categorization of my old emails – not an ideal scenario at all!

In this case, the category files might be the same size, so use your last modified date to give you an idea of which one you need to bring forward.


In order to import this Category list, make a copy of the one that you want to import, and then rename it to match the default one created by Outlook:


As you can see from the screenshot, I rename the default one to old, and then copy the name of the file to my clipboard. Rename your copy to match this (making sure that the file extension is still is .dat at the end), and voila!


Categories are back in place, and all is well with the world!

Update (July 2017): Going Deeper

After Kevin commented below on having multiple DAT files to try and sort through, I thought I’d take a look to see if there was a better way of figuring out which file holds your categories – trial and error is fine when there’s only a couple of files, but if you have a whole bunch, this gets annoying quickly. I had never tried opening these files in Notepad before, but essentially they’re just XML files, and so you can definitely open them up and find which files have your categories in them.

If I checked my current categories in Outlook, I can see I was actually missing these categories – I uninstalled and re-installed Office, and they ended up getting lost… perfect time to dig in and find my own fix!

So I opened both DAT files in Notepad ++ (if you don’t have it installed, you should – it’s amazing), and found my active file by finding the one that only had the MVP category:

I copy/pasted the lines that I wanted into my active DAT file – always make sure you keep the XML syntax intact – I noticed that my current category colour for MVP was “7”, or blue, which conflicted with my “Stratiform” category. I tried messing around a bit, changing the colour to orange, and replacing the orange line, but it wouldn’t stick when I saved the file:

So all I did from there was open the file back up again, undo my changes (love, love, love Notepad ++), and removed the orange line entirely, and added my MVP category back at the bottom, and all was well with the world again:

BOOM!

I’m sure smarter folks than I could tell me what specifically is happening in those XML files, and whether or not the order actually affects things – all I know is that my categories are back, and I’m in business again! 😀

 

Transfer Outlook 2010 Autocomplete Cache to a New Profile

One issue that can happen when creating a new Outlook profile in order to configure Office 365 access is that the Autocomplete cache disappears – the reason for this is that the autocomplete cache is tied to the old profile, and doesn’t get carried over automatically… the good news is that there is a way to import it from the old profile, so all is not lost!

Before:


Start by navigating to c:\Users\username\AppData\Local\Microsoft\Outlook\RoamCache – look for a file named Stream_Autocomplete_0.dat with a hash of numbers. If you have multiple profiles, you can sort by date, and choose the most recent one – the empty autocomplete file will typically be 1 – 2KB, so it’s usually pretty easy to see which one it is.


Next, find the Autocomplete file that you want to import – it’ll usually be quite a bit bigger, and it will have a different hash of numbers in the file name (and usually a different modified date as well):


In order for this to work, Outlook needs to be closed – once this is done, make a backup of the Autocomplete file that you plan to import – just in case. I typically make a copy of this file, and then work with the copy, so I can always go back to my original if I need to.

Next, rename the empty autocomplete dat file – just change the name of it to .bak, or .old, and that should be sufficient. At the same time, grab the name of the Autocomplete file that you want to replace (in this case, the name you’re copying is Stream_Autocomplete_0_A7D60F3ACC828B4EB204A03004F8BD58, and then rename the copy of the file you just made

Once you’re done, you should now have two autocomplete files of the same size – the autocomplete file from the original profile, and the one you’ve just renamed / imported:


Now, go ahead and open Outlook and verify that autocomplete is working properly again.

After:


There have been a few times that I’ve done this, and found my autocomplete file cleared again when I re-open Outlook – if this is the case, just do the process again. This is why we made a copy of the good autocomplete file, as we can still go back and redo the process – otherwise, your working copy of the autocomplete cache would be all gone, and that would be the end of it!

Sit back and relax, and get used to being hailed as a hero… This trick is a particular brand of magic that makes you seem like both a magician, and a miracle worker!

.TrimEnd removes too many characters

I’m working on a migration project where I need to create temporary accounts for each user that I’m going to be migrating (long story, don’t ask!). I wanted a way to create the temporary account based on the real user name, have them easily identifiable as belonging to that user, and then make sure to not use the primary domain for their email address, just to make sure there was no confusion.

Based on these requirements, I started working on a script to provision these user accounts – I wanted to take a user’s name and UPN from a CSV file, and then produce the temporary migration account from there.

For example, my csv file looked like this:

Name samAccountName UPN LicenseType UsageLocation
YVR E1 Test yvrE1test yvrE1test@masterandcmdr.com E1 CA

Just so you can follow along, I’ve imported the CSV file into my Shell so we can work with it:

Now that I have my variable defined, I needed to get just the beginning of the UPN, so I could create a new user. I know what you’re thinking – why not just use the samAccountName, since it matches? Well, I wanted to make sure I wouldn’t end up with discrepancies if I ran this against a larger batch of users, and had some where those values didn’t match – I figured the safest bet would be take the UPN value that I’d be using later (for the real user account), and build off of that.

So, I started out by using the .TrimEnd method to remove the domain name from the end of the UPN, like so:

$migUser = $($u.upn).TrimEnd(“@masterandcmdr.com”)

And after that, add a prefix, and the onmicrosoft domain to create a new UPN:

$migUPN = “mc-$migUser+“@masterandcmdr.onmicrosoft.com”

And finally, I wanted the Display Name to make it obvious that this was my temporary migration account:

$migDisplay = $($u.Name) (MC)”

What happened next was really weird – .TrimEnd was taking away more characters than I had expected, like so:


 

So the end result was that my user would be created, but the results were inconsistent – very frustrating!

Doing some digging around on the internet I discovered that TrimEnd treats the characters that you specify as a character array, and not a string like I was expecting it to. Since all of the letters for “test” are found in “masterandcmdr.com”, it was trimming away every character that it found at the end of the string that matched ANY of those characters. As soon as it hits a character that doesn’t match the array of characters you’ve provided, it stops trimming, which is why it doesn’t take away the remaining “rE” from my username.

To solve this problem, and to make sure that you are removing a specific string of text from the end of a word, use the -replace function instead, like so:

# Define migration user account format

$migUser = $($u.upn) -replace ‘@masterandcmdr.com’,

$migUPN = “mc-$migUser+“@masterandcmdr.onmicrosoft.com”

$migDisplay = $($u.Name) (MC)”

As you can see, this time my results were exactly as expected:


 

So, lessons learned – make sure if you need to remove a specific string of characters from the end of a string in PowerShell, use -replace and not .TrimEnd!

Distribution Groups, Naming Policies and You!

Office 365 Groups: Next Gen Distribution Lists?


Lately Microsoft has been putting a lot of focus on Office 365 groups as an ad hoc, user driven collaboration platform. These Office 365 Groups are also used for Microsoft Planner, as each Office 365 Group creates a plan, and every time a user creates a new plan a group is spun up in the background to handle all the collaboration and messaging pieces. Even going into the Exchange Online Portal and creating a new distribution groups will create an Office 365 Group by default – you need to select the option to create a regular distribution list instead.


These groups perform the job they were designed for quite admirably, and I’m a big fan of the user experience and control – however, where I feel these Groups are lacking is in the admin controls. To date, there is no way to export that mailbox data if you need to archive or delete the group, which makes it a pretty big gap in management (in my opinion at least).

Self Service: A Two-Edged Sword

One of the big selling features of these groups is that users can create their own groups – either in Outlook 2016 or Outlook on the Web. Now, this feature is great for allowing users some of the control that IT typically owns, and allowing them to quickly get some collaboration going – the downside is that it’s harder for IT to control and manage, and your directory can quickly become messy with groups users are creating to just test things out, or play around with the features. Thankfully, Microsoft has recently added the capability for users to delete groups that they own (something that was missing when groups where introduced).

Group Naming Policy

In order to keep a reign on the chaos of users creating and deleting groups, admins can implement a group naming policy in EAC, which will help to at least standardize the group naming structure, and highlight a few keywords that you want to keep off the naming roster.

To configure your naming policy, log into the Exchange online portal (https://outlook.office365.com/ecp), navigate to recipients – groups, and then click on the three dots to open up the context menu.

Click on Configure group naming policy:


Your first option is a prefix, which can be either an Attribute or Text:


One idea would be to prefix these user-created groups with an identifier, like “O365-“, but you can obviously make this whatever you want.


And then again, you can add suffix(es) if you want – again, you can use whatever you want, but an idea would be to use the city attribute of the user creating the group:


This policy will apply to all user created groups, whether created in Outlook or OWA – groups created from the admin portal will bypass this setting.

The Problem with Synced Groups

Oddly enough however, groups created through PowerShell or DirSync will still end up with this naming policy applied. This can become a problem, because a distribution group created on premise might be named “My New Group”, while the synced group will be named “O365-My New Group-Vancouver” (or whatever your policy is).

Here’s how you get around that problem:


<# .SYNOPSIS Script to create distribution groups and bypass the Exchange Online group naming policy. .PARAMETER GroupName This parameter is required - if spaces are required in the Group name, make sure to put the name in quotes. .NOTES File Name : create-DistributionGroup.ps1 Author : Jeremy Dahl (jdahl@masterandcmdr.com) .EXAMPLE .\create-DistributionGroup.ps1 -GroupName MyGroup Creates a group named "MyGroup", with a primary SMTP address of mygroup@mydomain.com .EXAMPLE .\create-DistributionGroup.ps1 -GroupName "My Group" Creates a group named "My Group", with a primary SMTP address of mygroup@mydomain.com #>
 
param (
    [Parameter(Mandatory=$true,ValueFromPipeline=$false)]
    [string] $GroupName = ""
    )

$smtpDomain="@mydomain.com" # Change this field to match your smtp domain
$exchangeServer="ExchangeServer" # Input your on premise Exchange Server here
$aadConnectServer="AADConnectServer" # Input your AAD Connect Server here
$GroupOU="OU=Managed Groups,DC=mydomain,DC=com" # Pick an OU for your groups to be created into - can be moved once the group is synced up.

$exchangeURI="http://$exchangeServer/PowerShell/"
$primarySMTP = $GroupName + $smtpDomain
 
# -- Connect to Office 365 -- #
$credential = Get-Credential
Connect-MsolService -Credential $credential
$ExchangeSession = New-PSSession -ConfigurationName Microsoft.Exchange -ConnectionUri https://outlook.office365.com/powershell-liveid/ -Credential $credential -Authentication Basic -AllowRedirection
$importresults = Import-PSSession $ExchangeSession -AllowClobber
 
<# -- Create Group in Exchange Online -- #>
New-DistributionGroup -Name $GroupName -DisplayName $GroupName -PrimarySmtpAddress $primarySMTP -IgnoreNamingPolicy
Remove-PSSession $ExchangeSession
 
 
<# -- Create a local Exchange session and import session for use -- #>
$LocalSession = New-PSSession -ConfigurationName Microsoft.Exchange -ConnectionUri $exchangeURI -Authentication Kerberos
Import-PSSession $LocalSession -AllowClobber
 
<# -- Create Group On Premise -- #>
New-DistributionGroup -Name $GroupName -OrganizationalUnit $GroupOU

<# -- Get Credentials and run AADSync remotely -- #>
$adCreds=Get-Credential
Invoke-Command -ComputerName $aadConnectServer -ScriptBlock {Start-ADSyncSyncCycle -PolicyType Delta} -Credential $adCreds
Write-Host "Initiated Azure AD Sync - Delta" -ForegroundColor Green

Download the script: 

This script can be run on premise, and only requires the Group Name as a parameter. It then connects to Exchange Online and creates the group, ignoring the naming policy. From there, it connects to Exchange on premise, and creates the same group, using the same group name. Once AAD Sync runs, it matches the group together, and treats it as a single group going forward.

 

Once the groups have synced up, I’ve confirmed that you can add members to it from on premise as normal, and even delete it on premise (removing it from the cloud as well) if necessary.

That’s it – problem solved!


On Premise attributes not updating properly

I’ve run into this problem a few times during recent migrations, so I’ve started incorporating the script blocks below into my migration scripts to make sure that I have a consistent experience with moving shared mailboxes, and enabling archive mailboxes.

Issue:

Sometimes msExchange attributes are not properly updated when a mailbox is moved in Exchange. This affects shared mailboxes, as well as creating archives on user mailboxes: https://support.microsoft.com/en-us/kb/2710029. This issue can also cause the mailboxes to show up with the wrong icons in the Exchange console on prem, which can be quite confusing!

Resolution:

Use the following script blocks to update the msExchRemoteRecipientType and the msExchangeRecipientTypeDetails – once these attributes are updated and DirSync has run, mailboxes will show up properly both on premise, and in Office 365. Update $samAccountName with the username you want to update.

Update user attributes to enable archiving:

Set-AdUser $SamAccountName -Replace @{msExchRemoteRecipientType=“3”}

Set-AdUser $SamAccountName -Replace @{msExchRecipientTypeDetails=“2147483648”}

Update AD attributes to disable user account and convert to a shared mailbox:

Disable-AdAccount $SamAccountName

Set-AdUser $SamAccountName -Replace @{msExchRemoteRecipientType=“100”}

Set-AdUser $SamAccountName -Replace @{msExchRecipientTypeDetails=“34359738368”}

This article was a great resource for listing all the different recipient types: http://memphistech.net/?p=457. Hope this helps you solve any mailbox weirdness you run into during migrations!

RetentionHoldEnabled Inconsistency

I ran into an issue today where we discovered that a subset of our users with mailboxes that had been migrated to Office 365 had Retention Hold enabled on their mailboxes – what was strange about this was that we hadn’t set this at all during our migration, and it seemed to be randomly applied to about 30% of the mailboxes.

You can check this setting with the following command:

Get-Mailbox -ResultSize unlimited | Where-Object {$_.RetentionHoldEnabled -eq $true} | Format-Table Name,RetentionPolicy,RetentionHoldEnabled –Auto

With this result:

2015-11-03_11-10-39

Looking through the list of users, there was a mixture of E1 and E3 licenses, but no K1 licenses or shared mailboxes. This made sense, as K1s and shared mailboxes didn’t have archiving enabled.

It’s important to note that Retention Hold is not the same as Litigation Hold – Litigation Hold puts a change freeze on a mailbox so that a user can’t delete or change items in their mailbox. It generally happens behind the scenes, and most users don’t notice that their mailbox has litigation hold applied, as deleted items disappear as normal, and the deletions/changes end up in a separate folder that the user cannot see.

Retention Hold, on the other hand, prevents the Managed Folder Assistant from running on that mailbox and processing retention tags. This means that users with Retention Hold enabled will not have their emails archiving or deleting based on the policies that have been set up in their Retention Policy.

After looking around a bit, I began to notice a pattern – each of the users who had their RetentionHold set to Enabled were users that we had been importing PST files into their online archives using the Office 365 Import Service. We had already noticed a bug (and opened a ticket), because we were not able to manually delete the jobs, and they weren’t automatically being deleted after 30 days the way they’re supposed to be.

It seems like Retention Hold is being enabled when a PST import job starts, and the flag is not being cleared automatically because the jobs are not being deleted properly.

You can fix this on a single mailbox by running the following command:

Set-Mailbox username@domain.com -RetentionHoldEnabled $false

Alternately, if you want to run this for all affected mailboxes, here’s the command to use:

Get-Mailbox -ResultSize unlimited | Where-Object {$_.RetentionHoldEnabled -eq $true} | Set-Mailbox -RetentionHoldEnabled $false

Hopefully Microsoft will get this bug resolved soon so that we have better control over the PST Import service – in the meantime, you can use this script to get retention policies functioning properly again.

Hope this helps!

Migrating to Office 365: The Good, The Bad, and The Ugly

Last night marked the launch of a new Cloud / Office 365 user group hosted by the team at Long View Systems – I had the pleasure of being the inaugural speaker, and we ended up having a great discussion around the various migration strategies available when moving email into Exchange Online.

It was awesome getting feedback from the folks that were at the meeting, and we had enough field experience in the room to be able to share our war stories, and talk about the good, bad, and ugly parts that make up the move into Microsoft’s cloud.

Hope you enjoy the show! 😀

Find which server DirSync is installed on

As consultants, we often find ourselves in environments that haven’t been properly managed, and usually not properly documented – which is generally why we get called in.

In this particular situation, I was asked to install and configure DirSync / AAD Connect for a client – only to find out that it was already installed! The funny part is that the client didn’t know it was installed, and nobody knew what server it was installed on, so I had to do some digging to find it.

Before logging in to every server and checking running services, installed programs, etc., I thought I’d take a look to see if there was a better way. Sure enough, fellow MVP Benoit Hamet suggested this answer on the Office 365 community forums. All props go to him for providing this simple solution:

“The MSOL account has a description which contains the server name on which it’s been installed”

So I checked in AD, and sure enough, this is what I found:

2015-10-16_9-43-50

And in the description?

“Account created by the Windows Azure Active Directory Sync tool with installation identifier ‘e80ac210a5e14d6095c0fcea79acc5f9’ running on computer ‘RANDOMSERVER‘ configured to synchronize to tenant ‘domain.com‘. This account must have directory replication permissions in the local Active Directory and write permission on certain attributes to enable Hybrid Deployment.”

Brilliant – thanks Benoit!

Enable OneDrive domain sync restrictions

One of the admin controls that has recently been added to OneDrive for Business is the ability to restrict file sync to only work on domain joined machines. Here’s how you enable this:

First, you need to get the domain GUID by running the following command in PowerShell:

$domains = (Get-ADForest).Domains; foreach($d in $domains) {Get-ADDomain -Identity $d | Select ObjectGuid}


Next, set the domain GUID as the only accepted domain for OneDrive sync:

Set-SPOTenantSyncClientRestriction  -Enable -DomainGuids "xxxxxxx-xxxx-415c-aa3b-9d06b595c714"

That’s really all there is to it – if you need to undo these changes and open sync back up again, simply run the following command:

Remove OneDrive domain sync restrictions:

Remove-SPOTenantSyncClientRestriction

When this feature is enabled the following will occur: (pulled directly from the TechNet article)

  • All OneDrive for Business Sync client requests originating from a domain that is not on the safe recipients list will be blocked.
  • All OneDrive for Business Mac Sync client requests will be blocked.
  • Mobile clients are not blocked when this feature is enabled.
  • Regardless whether a computer is managed by a device management solution, a sync relationship will not be established unless they are joined to a domain in the Safe Recipient List.
  • Any files that have been previously been synced down to your computer will not be deleted.
  • Please be aware the following upload behavior:
    • New or existing files added to the client will still be uploaded to the server and will not be blocked.
      • Regardless if the computer is joined to a domain which is set in the Safe Recipient List.
      • Regardless if the computer is joined to a domain which is not set in the Safe Recipient List.
      • And for all non-domain joined computers.
  • OneDrive for Business sync client prior to version 15.0.4693.1000 will stop syncing existing libraries.

 

For more information, see the following articles:
How to enumerate a domain GUID in an Active Directory forest: https://technet.microsoft.com/en-us/library/dn938435.aspx
Use Windows PowerShell cmdlets to enable OneDrive sync for domains that are on the safe recipients list: https://technet.microsoft.com/en-us/library/dn917452.aspx

Lessons Learned, SharePoint Online Edition

When you connect to as many different Office 365 tenancies as I do, it’s easy to lose track of which tenant you’re connecting to – especially if you’re working on one project, but logging on to another tenant to test changes.

Sure enough, I was testing OneDrive sync restrictions for a client, and I accidentally ran the cmdlets in their production tenant, and not in my test tenant like I thought. Basically, the purpose of the cmdlet is to only allow domains on a safe list to sync – everything else (including Macs) gets blocked.

Needless to say, I was horrified when users company-wide started reporting that their OneDrive sync had stopped working! Thankfully, I was able to reverse the changes fairly quickly, and my client took my mistake and profuse apologies in good grace.

Now, it’s one thing to call this a Lesson Learned, and say that you should always double check your tenant name before you connect to run PowerShell commands, but a good friend and colleague of mine has a saying about lessons learned that I’ve taken to heart:

“There are only three responses to lessons learned: either we need more research, give feedback to the consultant, or change the process.”

So I took my feedback like a big boy, and decided that I was going to figure out how to change my process to prevent these types of mistakes from happening again – and I re-wrote my connection script to force me to put in the tenant name every time I connected. It can’t completely prevent mistakes, but it at least prevents autodialing the wrong tenant if I’m not paying attention to what I’m doing.

Here’s what the script looks like:

param (
[Parameter(Mandatory=$true,ValueFromPipeline=$false)]
[String] $TenantName = ""
)

$spoDomain = "https://" + $TenantName
$spoDomain = $spoDomain + "-admin.sharepoint.com"
$objCreds = Get-Credential

Connect-SPOService -Url $spoDomain -credential $objCreds
Connect-MSOLService -credential $objCreds

 

And here’s a copy that you can download and use if you’d like: Connect-SharePoint-Online.ps1. Simply run this script with the -TenantName parameter, like so:

.\connect-SharePoint-Online.ps1 -TenantName Contoso

If you only ever connect to a single tenant, all you need to do is change your connection string to look like this:

$objCreds = Get-Credential
$spoDomain = "https://contoso-admin.sharepoint.com"

Connect-SPOService -Url $spoDomain -credential $objCreds
Connect-MSOLService -credential $objCreds 

This connects you to both SharePoint Online, as well as the MSOL Service so you can query / manage AD objects as well.

Hope this helps… as always, scripts or cmdlets are provided without any guarantees on my part – read it over and make sure you know what you’re running before you execute scripts in a production environment!