Master & Cmd-R

DirSync error: Attribute Value Must Be Unique

I’ve run into this error before, and it’s usually pretty straightforward – find the duplicate proxy address (usually), and remove it… problem solved.

This time, however, wasn’t as straightforward – searching for the duplicate proxy address that DirSync was complaining about turned up nothing at all in Active Directory!

I dug a bit deeper into the attributes for one of the affected users, and realized that they actually were missing the proxy address, not that it was duplicated at all. The reason for this is that these users were contractors, so their Primary SMTP address was, and even though their UPN was, there was no SMTP address for the primary domain.

Adding the missing proxy address resolved the problem immediately, and the users are now properly synced up, and my DirSync logs are clean again!

Hope this helps someone else who gets stuck wondering how no proxy address = duplicate proxy adress!

Unable to change Deleted Item Retention

I recently needed to update the Deleted Item Retention period in Office 365 from the default 14 days to the maximum allowed (30 days) for all mailboxes in my environment. Since I was migrating mailboxes to Office 365 at the time, I wrote a script that I could add to my process which would update this setting while it was applying quotas to the mailboxes.

Things were working well, apart from a number of Room mailboxes that had been migrated from Exchange on Premise – every time the script ran, I’d get the following warning on all these mailboxes:

The strange thing is that this was only happening for Rooms that were migrated from Exchange on Premise – any new rooms that were created didn’t have this issue. I decided to compare the mailbox attributes of a room that wasn’t affected by this issue to see what the difference was, and found this culprit:

UseDatabaseRetentionDefaults: True

Turning that setting off allowed me to go back and change the RetainDeletedItemsFor setting to 30 days, like I wanted to:

Set-Mailbox mailboxname -UseDatabaseRetentionDefaults $false

Set-Mailbox mailboxname -RetainDeletedItemsFor 30

In order to fix this for all other rooms affected by this issue, use the following command:

Get-Mailbox -ResultSize Unlimited | where {$_.ResourceType -eq “Room” -and $_.UseDatabaseRetentionDefaults -eq $true} | Set-Mailbox -UseDatabaseRetentionDefaults $false

After that, it was a simple matter of re-running my script – the deleted item retention piece looks like this:

$t = New-TimeSpan -Days 14

$retMailboxes = Get-Mailbox -ResultSize unlimited | Where {($_.Name -notmatch “DiscoverySearch” -and $_.RetainDeletedItemsFor -eq $t)}

foreach ($r in $retMailboxes){
Set-Mailbox -Identity $r -RetainDeletedItemsFor 30

Write-Host “Deleted Item Retention for $($r.Name) successfully updated to 30 days” -ForegroundColor Green

Hope this helps someone else scratching their head trying to figure out why they’re unable to change the Deleted Item Retention Period on mailboxes!

How and when Clutter is enabled

This question has been bugging me since Clutter was launched, and I was happy to find this thread on the IT Pro Network that answered it. Clutter is one of those features that I take for granted now, but it’s definitely a question that comes up during migrations when users are starting to see it, and some aren’t (yet).

“Let me clarify the issue here:

Clutter is a learning system. It requires to have a certain lower limit of messages in the mailbox to confidently learn about a user’s behavior before Clutter is auto enabled for a mailbox.

For newly created mailboxes and mailboxes that are migrated from On-Prem to the cloud, we need the following requirements to be satisfied:

1) At least 1000 messages delivered to the mailbox after creation (or migration to the cloud).

2) User needs to have logged into the mailbox once after creation (or migration to the cloud).

After the above two criteria are satisfied, Clutter is auto enabled for that mailbox within 24 hours.”

From <>

Restoring Outlook Categories

Similar to Autocomplete settings, Outlook Categories are saved in a Stream_CategoryList_1_guid.dat file. The GUID is random, and looks something like this:

In order to restore Outlook Categories when creating a new profile, you can use the same approach required for restoring Autocomplete files. Basically, rename the old categories .dat file to match the DAT file associated with your new profile.

Here’s what my categories looked like before creating a new profile:

And here is the generic categories that Outlook comes with:

Not only am I missing my custom categories which prevents me from setting the categories I want on all my new emails, I’m also missing the categorization of my old emails – not an ideal scenario at all!

In this case, the category files might be the same size, so use your last modified date to give you an idea of which one you need to bring forward.

In order to import this Category list, make a copy of the one that you want to import, and then rename it to match the default one created by Outlook:

As you can see from the screenshot, I rename the default one to old, and then copy the name of the file to my clipboard. Rename your copy to match this (making sure that the file extension is still is .dat at the end), and voila!

Categories are back in place, and all is well with the world!

Update (July 2017): Going Deeper

After Kevin commented below on having multiple DAT files to try and sort through, I thought I’d take a look to see if there was a better way of figuring out which file holds your categories – trial and error is fine when there’s only a couple of files, but if you have a whole bunch, this gets annoying quickly. I had never tried opening these files in Notepad before, but essentially they’re just XML files, and so you can definitely open them up and find which files have your categories in them.

If I checked my current categories in Outlook, I can see I was actually missing these categories – I uninstalled and re-installed Office, and they ended up getting lost… perfect time to dig in and find my own fix!

So I opened both DAT files in Notepad ++ (if you don’t have it installed, you should – it’s amazing), and found my active file by finding the one that only had the MVP category:

I copy/pasted the lines that I wanted into my active DAT file – always make sure you keep the XML syntax intact – I noticed that my current category colour for MVP was “7”, or blue, which conflicted with my “Stratiform” category. I tried messing around a bit, changing the colour to orange, and replacing the orange line, but it wouldn’t stick when I saved the file:

So all I did from there was open the file back up again, undo my changes (love, love, love Notepad ++), and removed the orange line entirely, and added my MVP category back at the bottom, and all was well with the world again:


I’m sure smarter folks than I could tell me what specifically is happening in those XML files, and whether or not the order actually affects things – all I know is that my categories are back, and I’m in business again! 😀


Transfer Outlook 2010 Autocomplete Cache to a New Profile

One issue that can happen when creating a new Outlook profile in order to configure Office 365 access is that the Autocomplete cache disappears – the reason for this is that the autocomplete cache is tied to the old profile, and doesn’t get carried over automatically… the good news is that there is a way to import it from the old profile, so all is not lost!


Start by navigating to c:\Users\username\AppData\Local\Microsoft\Outlook\RoamCache – look for a file named Stream_Autocomplete_0.dat with a hash of numbers. If you have multiple profiles, you can sort by date, and choose the most recent one – the empty autocomplete file will typically be 1 – 2KB, so it’s usually pretty easy to see which one it is.

Next, find the Autocomplete file that you want to import – it’ll usually be quite a bit bigger, and it will have a different hash of numbers in the file name (and usually a different modified date as well):

In order for this to work, Outlook needs to be closed – once this is done, make a backup of the Autocomplete file that you plan to import – just in case. I typically make a copy of this file, and then work with the copy, so I can always go back to my original if I need to.

Next, rename the empty autocomplete dat file – just change the name of it to .bak, or .old, and that should be sufficient. At the same time, grab the name of the Autocomplete file that you want to replace (in this case, the name you’re copying is Stream_Autocomplete_0_A7D60F3ACC828B4EB204A03004F8BD58, and then rename the copy of the file you just made

Once you’re done, you should now have two autocomplete files of the same size – the autocomplete file from the original profile, and the one you’ve just renamed / imported:

Now, go ahead and open Outlook and verify that autocomplete is working properly again.


There have been a few times that I’ve done this, and found my autocomplete file cleared again when I re-open Outlook – if this is the case, just do the process again. This is why we made a copy of the good autocomplete file, as we can still go back and redo the process – otherwise, your working copy of the autocomplete cache would be all gone, and that would be the end of it!

Sit back and relax, and get used to being hailed as a hero… This trick is a particular brand of magic that makes you seem like both a magician, and a miracle worker!

.TrimEnd removes too many characters

I’m working on a migration project where I need to create temporary accounts for each user that I’m going to be migrating (long story, don’t ask!). I wanted a way to create the temporary account based on the real user name, have them easily identifiable as belonging to that user, and then make sure to not use the primary domain for their email address, just to make sure there was no confusion.

Based on these requirements, I started working on a script to provision these user accounts – I wanted to take a user’s name and UPN from a CSV file, and then produce the temporary migration account from there.

For example, my csv file looked like this:

Name samAccountName UPN LicenseType UsageLocation
YVR E1 Test yvrE1test E1 CA

Just so you can follow along, I’ve imported the CSV file into my Shell so we can work with it:

Now that I have my variable defined, I needed to get just the beginning of the UPN, so I could create a new user. I know what you’re thinking – why not just use the samAccountName, since it matches? Well, I wanted to make sure I wouldn’t end up with discrepancies if I ran this against a larger batch of users, and had some where those values didn’t match – I figured the safest bet would be take the UPN value that I’d be using later (for the real user account), and build off of that.

So, I started out by using the .TrimEnd method to remove the domain name from the end of the UPN, like so:

$migUser = $($u.upn).TrimEnd(“”)

And after that, add a prefix, and the onmicrosoft domain to create a new UPN:

$migUPN = “mc-$migUser+“”

And finally, I wanted the Display Name to make it obvious that this was my temporary migration account:

$migDisplay = $($u.Name) (MC)”

What happened next was really weird – .TrimEnd was taking away more characters than I had expected, like so:


So the end result was that my user would be created, but the results were inconsistent – very frustrating!

Doing some digging around on the internet I discovered that TrimEnd treats the characters that you specify as a character array, and not a string like I was expecting it to. Since all of the letters for “test” are found in “”, it was trimming away every character that it found at the end of the string that matched ANY of those characters. As soon as it hits a character that doesn’t match the array of characters you’ve provided, it stops trimming, which is why it doesn’t take away the remaining “rE” from my username.

To solve this problem, and to make sure that you are removing a specific string of text from the end of a word, use the -replace function instead, like so:

# Define migration user account format

$migUser = $($u.upn) -replace ‘’,

$migUPN = “mc-$migUser+“”

$migDisplay = $($u.Name) (MC)”

As you can see, this time my results were exactly as expected:


So, lessons learned – make sure if you need to remove a specific string of characters from the end of a string in PowerShell, use -replace and not .TrimEnd!

Distribution Groups, Naming Policies and You!

Office 365 Groups: Next Gen Distribution Lists?

Lately Microsoft has been putting a lot of focus on Office 365 groups as an ad hoc, user driven collaboration platform. These Office 365 Groups are also used for Microsoft Planner, as each Office 365 Group creates a plan, and every time a user creates a new plan a group is spun up in the background to handle all the collaboration and messaging pieces. Even going into the Exchange Online Portal and creating a new distribution groups will create an Office 365 Group by default – you need to select the option to create a regular distribution list instead.

These groups perform the job they were designed for quite admirably, and I’m a big fan of the user experience and control – however, where I feel these Groups are lacking is in the admin controls. To date, there is no way to export that mailbox data if you need to archive or delete the group, which makes it a pretty big gap in management (in my opinion at least).

Self Service: A Two-Edged Sword

One of the big selling features of these groups is that users can create their own groups – either in Outlook 2016 or Outlook on the Web. Now, this feature is great for allowing users some of the control that IT typically owns, and allowing them to quickly get some collaboration going – the downside is that it’s harder for IT to control and manage, and your directory can quickly become messy with groups users are creating to just test things out, or play around with the features. Thankfully, Microsoft has recently added the capability for users to delete groups that they own (something that was missing when groups where introduced).

Group Naming Policy

In order to keep a reign on the chaos of users creating and deleting groups, admins can implement a group naming policy in EAC, which will help to at least standardize the group naming structure, and highlight a few keywords that you want to keep off the naming roster.

To configure your naming policy, log into the Exchange online portal (, navigate to recipients – groups, and then click on the three dots to open up the context menu.

Click on Configure group naming policy:

Your first option is a prefix, which can be either an Attribute or Text:

One idea would be to prefix these user-created groups with an identifier, like “O365-“, but you can obviously make this whatever you want.

And then again, you can add suffix(es) if you want – again, you can use whatever you want, but an idea would be to use the city attribute of the user creating the group:

This policy will apply to all user created groups, whether created in Outlook or OWA – groups created from the admin portal will bypass this setting.

The Problem with Synced Groups

Oddly enough however, groups created through PowerShell or DirSync will still end up with this naming policy applied. This can become a problem, because a distribution group created on premise might be named “My New Group”, while the synced group will be named “O365-My New Group-Vancouver” (or whatever your policy is).

Here’s how you get around that problem:

<# .SYNOPSIS Script to create distribution groups and bypass the Exchange Online group naming policy. .PARAMETER GroupName This parameter is required - if spaces are required in the Group name, make sure to put the name in quotes. .NOTES File Name : create-DistributionGroup.ps1 Author : Jeremy Dahl ( .EXAMPLE .\create-DistributionGroup.ps1 -GroupName MyGroup Creates a group named "MyGroup", with a primary SMTP address of .EXAMPLE .\create-DistributionGroup.ps1 -GroupName "My Group" Creates a group named "My Group", with a primary SMTP address of #>
param (
    [string] $GroupName = ""

$smtpDomain="" # Change this field to match your smtp domain
$exchangeServer="ExchangeServer" # Input your on premise Exchange Server here
$aadConnectServer="AADConnectServer" # Input your AAD Connect Server here
$GroupOU="OU=Managed Groups,DC=mydomain,DC=com" # Pick an OU for your groups to be created into - can be moved once the group is synced up.

$primarySMTP = $GroupName + $smtpDomain
# -- Connect to Office 365 -- #
$credential = Get-Credential
Connect-MsolService -Credential $credential
$ExchangeSession = New-PSSession -ConfigurationName Microsoft.Exchange -ConnectionUri -Credential $credential -Authentication Basic -AllowRedirection
$importresults = Import-PSSession $ExchangeSession -AllowClobber
<# -- Create Group in Exchange Online -- #>
New-DistributionGroup -Name $GroupName -DisplayName $GroupName -PrimarySmtpAddress $primarySMTP -IgnoreNamingPolicy
Remove-PSSession $ExchangeSession
<# -- Create a local Exchange session and import session for use -- #>
$LocalSession = New-PSSession -ConfigurationName Microsoft.Exchange -ConnectionUri $exchangeURI -Authentication Kerberos
Import-PSSession $LocalSession -AllowClobber
<# -- Create Group On Premise -- #>
New-DistributionGroup -Name $GroupName -OrganizationalUnit $GroupOU

<# -- Get Credentials and run AADSync remotely -- #>
Invoke-Command -ComputerName $aadConnectServer -ScriptBlock {Start-ADSyncSyncCycle -PolicyType Delta} -Credential $adCreds
Write-Host "Initiated Azure AD Sync - Delta" -ForegroundColor Green

Download the script: 

This script can be run on premise, and only requires the Group Name as a parameter. It then connects to Exchange Online and creates the group, ignoring the naming policy. From there, it connects to Exchange on premise, and creates the same group, using the same group name. Once AAD Sync runs, it matches the group together, and treats it as a single group going forward.


Once the groups have synced up, I’ve confirmed that you can add members to it from on premise as normal, and even delete it on premise (removing it from the cloud as well) if necessary.

That’s it – problem solved!

On Premise attributes not updating properly

I’ve run into this problem a few times during recent migrations, so I’ve started incorporating the script blocks below into my migration scripts to make sure that I have a consistent experience with moving shared mailboxes, and enabling archive mailboxes.


Sometimes msExchange attributes are not properly updated when a mailbox is moved in Exchange. This affects shared mailboxes, as well as creating archives on user mailboxes: This issue can also cause the mailboxes to show up with the wrong icons in the Exchange console on prem, which can be quite confusing!


Use the following script blocks to update the msExchRemoteRecipientType and the msExchangeRecipientTypeDetails – once these attributes are updated and DirSync has run, mailboxes will show up properly both on premise, and in Office 365. Update $samAccountName with the username you want to update.

Update user attributes to enable archiving:

Set-AdUser $SamAccountName -Replace @{msExchRemoteRecipientType=“3”}

Set-AdUser $SamAccountName -Replace @{msExchRecipientTypeDetails=“2147483648”}

Update AD attributes to disable user account and convert to a shared mailbox:

Disable-AdAccount $SamAccountName

Set-AdUser $SamAccountName -Replace @{msExchRemoteRecipientType=“100”}

Set-AdUser $SamAccountName -Replace @{msExchRecipientTypeDetails=“34359738368”}

This article was a great resource for listing all the different recipient types: Hope this helps you solve any mailbox weirdness you run into during migrations!

RetentionHoldEnabled Inconsistency

I ran into an issue today where we discovered that a subset of our users with mailboxes that had been migrated to Office 365 had Retention Hold enabled on their mailboxes – what was strange about this was that we hadn’t set this at all during our migration, and it seemed to be randomly applied to about 30% of the mailboxes.

You can check this setting with the following command:

Get-Mailbox -ResultSize unlimited | Where-Object {$_.RetentionHoldEnabled -eq $true} | Format-Table Name,RetentionPolicy,RetentionHoldEnabled –Auto

With this result:


Looking through the list of users, there was a mixture of E1 and E3 licenses, but no K1 licenses or shared mailboxes. This made sense, as K1s and shared mailboxes didn’t have archiving enabled.

It’s important to note that Retention Hold is not the same as Litigation Hold – Litigation Hold puts a change freeze on a mailbox so that a user can’t delete or change items in their mailbox. It generally happens behind the scenes, and most users don’t notice that their mailbox has litigation hold applied, as deleted items disappear as normal, and the deletions/changes end up in a separate folder that the user cannot see.

Retention Hold, on the other hand, prevents the Managed Folder Assistant from running on that mailbox and processing retention tags. This means that users with Retention Hold enabled will not have their emails archiving or deleting based on the policies that have been set up in their Retention Policy.

After looking around a bit, I began to notice a pattern – each of the users who had their RetentionHold set to Enabled were users that we had been importing PST files into their online archives using the Office 365 Import Service. We had already noticed a bug (and opened a ticket), because we were not able to manually delete the jobs, and they weren’t automatically being deleted after 30 days the way they’re supposed to be.

It seems like Retention Hold is being enabled when a PST import job starts, and the flag is not being cleared automatically because the jobs are not being deleted properly.

You can fix this on a single mailbox by running the following command:

Set-Mailbox -RetentionHoldEnabled $false

Alternately, if you want to run this for all affected mailboxes, here’s the command to use:

Get-Mailbox -ResultSize unlimited | Where-Object {$_.RetentionHoldEnabled -eq $true} | Set-Mailbox -RetentionHoldEnabled $false

Hopefully Microsoft will get this bug resolved soon so that we have better control over the PST Import service – in the meantime, you can use this script to get retention policies functioning properly again.

Hope this helps!

Migrating to Office 365: The Good, The Bad, and The Ugly

Last night marked the launch of a new Cloud / Office 365 user group hosted by the team at Long View Systems – I had the pleasure of being the inaugural speaker, and we ended up having a great discussion around the various migration strategies available when moving email into Exchange Online.

It was awesome getting feedback from the folks that were at the meeting, and we had enough field experience in the room to be able to share our war stories, and talk about the good, bad, and ugly parts that make up the move into Microsoft’s cloud.

Hope you enjoy the show! 😀