Master & Cmd-R

Inbox rules: Inconsistent Forwarding

The issue: Inconsistent Forwarding Rules

I ran into an interesting issue on a recent migration: my client had a rather complicated forwarding scenario set up to manage a number of email subscriptions. Various folks around the (global) organization had signed up for these subscriptions, and they wanted to be able to distribute them throughout the company.

In order to facilitate this requirement, each subscription user had inbox rules set up to forward these subscriptions to a single mailbox, which then had around 50 inbox rules that were taking these emails and forwarding them on to a number of distribution groups, and then filing the email away in a folder – complicated, right?

First Rule: External Subscription to User:


Second Rule: Forward Subscription to Group:


Interestingly enough, this scenario worked in their old hosted environment, but once these mailboxes were migrated to Office 365 these rules started failing in a strange way: the rule would fire and move the message to the appropriate folder, but it wouldn’t actually forward the message on to the distribution group!

We tried recreating the rules, changing the wording of the rules, redirection vs. forwarding, etc. – nothing worked. Even weirder, if you tried to trigger the rule manually by sending an email from the source mailbox which met the rule criteria, the rule would trigger properly, making us think we actually had it solved this time. However, the next time the rule was activated automatically it would go back to the old behavior of filing but not forwarding.

So. Frustrating.

 

The Answer: “By Design”

Eventually I found the answer on the following forum post: Basically, as long as Exchange detects that the email was forwarded or redirected via a rule (mail headers include the field X-MS-Exchange-Inbox-Rules-Loop), it will not forward it a second time. This is by design in order to prevent unlimited loop forwarding.

This is also why we could trigger the rule manually by sending an email from the source mailbox – since the email was not forwarded it wouldn’t be caught by this behavior.

 

The Fix: Easier than you’d think

Thankfully, once we realized why this was happening, it was easy to resolve – simply remove the first forwarding rule (eliminating the loop potential), and instead replace it with a transport rule in Exchange Online.

For this rule, you can match the settings of the original inbox rule: If the sender is someone, do something. Depending on your requirements, you can choose to have it redirect the message, or simply Bcc the message to another user. Remember that if you choose redirection, the email will no longer go to the original user, so you might want to choose Bcc instead.


If you want to filter it better than this (the rule above might catch more emails than you want), simply click more options at the bottom of the rule window:


Once you have more options enabled, you can add as much details as you need to make sure you’re just catching the email you want, and not casting too wide a net:


Once you save your rule and activate it, make sure you disable or delete the original inbox rule so that you’re not generating duplicate emails, and then sit back and enjoy the magic!

DirSync error: Attribute Value Must Be Unique

I’ve run into this error before, and it’s usually pretty straightforward – find the duplicate proxy address (usually), and remove it… problem solved.


This time, however, wasn’t as straightforward – searching for the duplicate proxy address that DirSync was complaining about turned up nothing at all in Active Directory!

I dug a bit deeper into the attributes for one of the affected users, and realized that they actually were missing the proxy address, not that it was duplicated at all. The reason for this is that these users were contractors, so their Primary SMTP address was user@contractors.domain.com, and even though their UPN was user@domain.com, there was no SMTP address for the primary domain.

Adding the missing proxy address resolved the problem immediately, and the users are now properly synced up, and my DirSync logs are clean again!

Hope this helps someone else who gets stuck wondering how no proxy address = duplicate proxy adress!


Unable to change Deleted Item Retention

I recently needed to update the Deleted Item Retention period in Office 365 from the default 14 days to the maximum allowed (30 days) for all mailboxes in my environment. Since I was migrating mailboxes to Office 365 at the time, I wrote a script that I could add to my process which would update this setting while it was applying quotas to the mailboxes.

Things were working well, apart from a number of Room mailboxes that had been migrated from Exchange on Premise – every time the script ran, I’d get the following warning on all these mailboxes:


The strange thing is that this was only happening for Rooms that were migrated from Exchange on Premise – any new rooms that were created didn’t have this issue. I decided to compare the mailbox attributes of a room that wasn’t affected by this issue to see what the difference was, and found this culprit:

UseDatabaseRetentionDefaults: True

Turning that setting off allowed me to go back and change the RetainDeletedItemsFor setting to 30 days, like I wanted to:

Set-Mailbox mailboxname -UseDatabaseRetentionDefaults $false


Set-Mailbox mailboxname -RetainDeletedItemsFor 30


In order to fix this for all other rooms affected by this issue, use the following command:

Get-Mailbox -ResultSize Unlimited | where {$_.ResourceType -eq “Room” -and $_.UseDatabaseRetentionDefaults -eq $true} | Set-Mailbox -UseDatabaseRetentionDefaults $false

After that, it was a simple matter of re-running my script – the deleted item retention piece looks like this:

$t = New-TimeSpan -Days 14

$retMailboxes = Get-Mailbox -ResultSize unlimited | Where {($_.Name -notmatch “DiscoverySearch” -and $_.RetainDeletedItemsFor -eq $t)}

foreach ($r in $retMailboxes){
Set-Mailbox -Identity $r -RetainDeletedItemsFor 30

Write-Host “Deleted Item Retention for $($r.Name) successfully updated to 30 days” -ForegroundColor Green

Hope this helps someone else scratching their head trying to figure out why they’re unable to change the Deleted Item Retention Period on mailboxes!

How and when Clutter is enabled

This question has been bugging me since Clutter was launched, and I was happy to find this thread on the IT Pro Network that answered it. Clutter is one of those features that I take for granted now, but it’s definitely a question that comes up during migrations when users are starting to see it, and some aren’t (yet).

“Let me clarify the issue here:

Clutter is a learning system. It requires to have a certain lower limit of messages in the mailbox to confidently learn about a user’s behavior before Clutter is auto enabled for a mailbox.

For newly created mailboxes and mailboxes that are migrated from On-Prem to the cloud, we need the following requirements to be satisfied:

1) At least 1000 messages delivered to the mailbox after creation (or migration to the cloud).

2) User needs to have logged into the mailbox once after creation (or migration to the cloud).

After the above two criteria are satisfied, Clutter is auto enabled for that mailbox within 24 hours.”

From <https://www.yammer.com/itpronetwork/#/users/1523145278?trk_event=search_ac&trk_model=user&trk_len=5&trk_pos=0>


Transfer Outlook 2010 Autocomplete Cache to a New Profile

One issue that can happen when creating a new Outlook profile in order to configure Office 365 access is that the Autocomplete cache disappears – the reason for this is that the autocomplete cache is tied to the old profile, and doesn’t get carried over automatically… the good news is that there is a way to import it from the old profile, so all is not lost!

Before:


Start by navigating to c:\Users\username\AppData\Local\Microsoft\Outlook\RoamCache – look for a file named Stream_Autocomplete_0.dat with a hash of numbers. If you have multiple profiles, you can sort by date, and choose the most recent one – the empty autocomplete file will typically be 1 – 2KB, so it’s usually pretty easy to see which one it is.


Next, find the Autocomplete file that you want to import – it’ll usually be quite a bit bigger, and it will have a different hash of numbers in the file name (and usually a different modified date as well):


In order for this to work, Outlook needs to be closed – once this is done, make a backup of the Autocomplete file that you plan to import – just in case. I typically make a copy of this file, and then work with the copy, so I can always go back to my original if I need to.

Next, rename the empty autocomplete dat file – just change the name of it to .bak, or .old, and that should be sufficient. At the same time, grab the name of the Autocomplete file that you want to replace (in this case, the name you’re copying is Stream_Autocomplete_0_A7D60F3ACC828B4EB204A03004F8BD58, and then rename the copy of the file you just made

Once you’re done, you should now have two autocomplete files of the same size – the autocomplete file from the original profile, and the one you’ve just renamed / imported:


Now, go ahead and open Outlook and verify that autocomplete is working properly again.

After:


There have been a few times that I’ve done this, and found my autocomplete file cleared again when I re-open Outlook – if this is the case, just do the process again. This is why we made a copy of the good autocomplete file, as we can still go back and redo the process – otherwise, your working copy of the autocomplete cache would be all gone, and that would be the end of it!

Sit back and relax, and get used to being hailed as a hero… This trick is a particular brand of magic that makes you seem like both a magician, and a miracle worker!

.TrimEnd removes too many characters

I’m working on a migration project where I need to create temporary accounts for each user that I’m going to be migrating (long story, don’t ask!). I wanted a way to create the temporary account based on the real user name, have them easily identifiable as belonging to that user, and then make sure to not use the primary domain for their email address, just to make sure there was no confusion.

Based on these requirements, I started working on a script to provision these user accounts – I wanted to take a user’s name and UPN from a CSV file, and then produce the temporary migration account from there.

For example, my csv file looked like this:

Name samAccountName UPN LicenseType UsageLocation
YVR E1 Test yvrE1test yvrE1test@masterandcmdr.com E1 CA

Just so you can follow along, I’ve imported the CSV file into my Shell so we can work with it:

Now that I have my variable defined, I needed to get just the beginning of the UPN, so I could create a new user. I know what you’re thinking – why not just use the samAccountName, since it matches? Well, I wanted to make sure I wouldn’t end up with discrepancies if I ran this against a larger batch of users, and had some where those values didn’t match – I figured the safest bet would be take the UPN value that I’d be using later (for the real user account), and build off of that.

So, I started out by using the .TrimEnd method to remove the domain name from the end of the UPN, like so:

$migUser = $($u.upn).TrimEnd(“@masterandcmdr.com”)

And after that, add a prefix, and the onmicrosoft domain to create a new UPN:

$migUPN = “mc-$migUser+“@masterandcmdr.onmicrosoft.com”

And finally, I wanted the Display Name to make it obvious that this was my temporary migration account:

$migDisplay = $($u.Name) (MC)”

What happened next was really weird – .TrimEnd was taking away more characters than I had expected, like so:


 

So the end result was that my user would be created, but the results were inconsistent – very frustrating!

Doing some digging around on the internet I discovered that TrimEnd treats the characters that you specify as a character array, and not a string like I was expecting it to. Since all of the letters for “test” are found in “masterandcmdr.com”, it was trimming away every character that it found at the end of the string that matched ANY of those characters. As soon as it hits a character that doesn’t match the array of characters you’ve provided, it stops trimming, which is why it doesn’t take away the remaining “rE” from my username.

To solve this problem, and to make sure that you are removing a specific string of text from the end of a word, use the -replace function instead, like so:

# Define migration user account format

$migUser = $($u.upn) -replace ‘@masterandcmdr.com’,

$migUPN = “mc-$migUser+“@masterandcmdr.onmicrosoft.com”

$migDisplay = $($u.Name) (MC)”

As you can see, this time my results were exactly as expected:


 

So, lessons learned – make sure if you need to remove a specific string of characters from the end of a string in PowerShell, use -replace and not .TrimEnd!

Distribution Groups, Naming Policies and You!

Office 365 Groups: Next Gen Distribution Lists?


Lately Microsoft has been putting a lot of focus on Office 365 groups as an ad hoc, user driven collaboration platform. These Office 365 Groups are also used for Microsoft Planner, as each Office 365 Group creates a plan, and every time a user creates a new plan a group is spun up in the background to handle all the collaboration and messaging pieces. Even going into the Exchange Online Portal and creating a new distribution groups will create an Office 365 Group by default – you need to select the option to create a regular distribution list instead.


These groups perform the job they were designed for quite admirably, and I’m a big fan of the user experience and control – however, where I feel these Groups are lacking is in the admin controls. To date, there is no way to export that mailbox data if you need to archive or delete the group, which makes it a pretty big gap in management (in my opinion at least).

Self Service: A Two-Edged Sword

One of the big selling features of these groups is that users can create their own groups – either in Outlook 2016 or Outlook on the Web. Now, this feature is great for allowing users some of the control that IT typically owns, and allowing them to quickly get some collaboration going – the downside is that it’s harder for IT to control and manage, and your directory can quickly become messy with groups users are creating to just test things out, or play around with the features. Thankfully, Microsoft has recently added the capability for users to delete groups that they own (something that was missing when groups where introduced).

Group Naming Policy

In order to keep a reign on the chaos of users creating and deleting groups, admins can implement a group naming policy in EAC, which will help to at least standardize the group naming structure, and highlight a few keywords that you want to keep off the naming roster.

To configure your naming policy, log into the Exchange online portal (https://outlook.office365.com/ecp), navigate to recipients – groups, and then click on the three dots to open up the context menu.

Click on Configure group naming policy:


Your first option is a prefix, which can be either an Attribute or Text:


One idea would be to prefix these user-created groups with an identifier, like “O365-“, but you can obviously make this whatever you want.


And then again, you can add suffix(es) if you want – again, you can use whatever you want, but an idea would be to use the city attribute of the user creating the group:


This policy will apply to all user created groups, whether created in Outlook or OWA – groups created from the admin portal will bypass this setting.

The Problem with Synced Groups

Oddly enough however, groups created through PowerShell or DirSync will still end up with this naming policy applied. This can become a problem, because a distribution group created on premise might be named “My New Group”, while the synced group will be named “O365-My New Group-Vancouver” (or whatever your policy is).

Here’s how you get around that problem:


<# .SYNOPSIS Script to create distribution groups and bypass the Exchange Online group naming policy. .PARAMETER GroupName This parameter is required - if spaces are required in the Group name, make sure to put the name in quotes. .NOTES File Name : create-DistributionGroup.ps1 Author : Jeremy Dahl (jdahl@masterandcmdr.com) .EXAMPLE .\create-DistributionGroup.ps1 -GroupName MyGroup Creates a group named "MyGroup", with a primary SMTP address of mygroup@mydomain.com .EXAMPLE .\create-DistributionGroup.ps1 -GroupName "My Group" Creates a group named "My Group", with a primary SMTP address of mygroup@mydomain.com #>
 
param (
    [Parameter(Mandatory=$true,ValueFromPipeline=$false)]
    [string] $GroupName = ""
    )

$smtpDomain="@mydomain.com" # Change this field to match your smtp domain
$exchangeServer="ExchangeServer" # Input your on premise Exchange Server here
$aadConnectServer="AADConnectServer" # Input your AAD Connect Server here
$GroupOU="OU=Managed Groups,DC=mydomain,DC=com" # Pick an OU for your groups to be created into - can be moved once the group is synced up.

$exchangeURI="http://$exchangeServer/PowerShell/"
$primarySMTP = $GroupName + $smtpDomain
 
# -- Connect to Office 365 -- #
$credential = Get-Credential
Connect-MsolService -Credential $credential
$ExchangeSession = New-PSSession -ConfigurationName Microsoft.Exchange -ConnectionUri https://outlook.office365.com/powershell-liveid/ -Credential $credential -Authentication Basic -AllowRedirection
$importresults = Import-PSSession $ExchangeSession -AllowClobber
 
<# -- Create Group in Exchange Online -- #>
New-DistributionGroup -Name $GroupName -DisplayName $GroupName -PrimarySmtpAddress $primarySMTP -IgnoreNamingPolicy
Remove-PSSession $ExchangeSession
 
 
<# -- Create a local Exchange session and import session for use -- #>
$LocalSession = New-PSSession -ConfigurationName Microsoft.Exchange -ConnectionUri $exchangeURI -Authentication Kerberos
Import-PSSession $LocalSession -AllowClobber
 
<# -- Create Group On Premise -- #>
New-DistributionGroup -Name $GroupName -OrganizationalUnit $GroupOU

<# -- Get Credentials and run AADSync remotely -- #>
$adCreds=Get-Credential
Invoke-Command -ComputerName $aadConnectServer -ScriptBlock {Start-ADSyncSyncCycle -PolicyType Delta} -Credential $adCreds
Write-Host "Initiated Azure AD Sync - Delta" -ForegroundColor Green

Download the script: 

This script can be run on premise, and only requires the Group Name as a parameter. It then connects to Exchange Online and creates the group, ignoring the naming policy. From there, it connects to Exchange on premise, and creates the same group, using the same group name. Once AAD Sync runs, it matches the group together, and treats it as a single group going forward.

 

Once the groups have synced up, I’ve confirmed that you can add members to it from on premise as normal, and even delete it on premise (removing it from the cloud as well) if necessary.

That’s it – problem solved!


On Premise attributes not updating properly

I’ve run into this problem a few times during recent migrations, so I’ve started incorporating the script blocks below into my migration scripts to make sure that I have a consistent experience with moving shared mailboxes, and enabling archive mailboxes.

Issue:

Sometimes msExchange attributes are not properly updated when a mailbox is moved in Exchange. This affects shared mailboxes, as well as creating archives on user mailboxes: https://support.microsoft.com/en-us/kb/2710029. This issue can also cause the mailboxes to show up with the wrong icons in the Exchange console on prem, which can be quite confusing!

Resolution:

Use the following script blocks to update the msExchRemoteRecipientType and the msExchangeRecipientTypeDetails – once these attributes are updated and DirSync has run, mailboxes will show up properly both on premise, and in Office 365. Update $samAccountName with the username you want to update.

Update user attributes to enable archiving:

Set-AdUser $SamAccountName -Replace @{msExchRemoteRecipientType=“3”}

Set-AdUser $SamAccountName -Replace @{msExchRecipientTypeDetails=“2147483648”}

Update AD attributes to disable user account and convert to a shared mailbox:

Disable-AdAccount $SamAccountName

Set-AdUser $SamAccountName -Replace @{msExchRemoteRecipientType=“100”}

Set-AdUser $SamAccountName -Replace @{msExchRecipientTypeDetails=“34359738368”}

This article was a great resource for listing all the different recipient types: http://memphistech.net/?p=457. Hope this helps you solve any mailbox weirdness you run into during migrations!

RetentionHoldEnabled Inconsistency

I ran into an issue today where we discovered that a subset of our users with mailboxes that had been migrated to Office 365 had Retention Hold enabled on their mailboxes – what was strange about this was that we hadn’t set this at all during our migration, and it seemed to be randomly applied to about 30% of the mailboxes.

You can check this setting with the following command:

Get-Mailbox -ResultSize unlimited | Where-Object {$_.RetentionHoldEnabled -eq $true} | Format-Table Name,RetentionPolicy,RetentionHoldEnabled –Auto

With this result:

2015-11-03_11-10-39

Looking through the list of users, there was a mixture of E1 and E3 licenses, but no K1 licenses or shared mailboxes. This made sense, as K1s and shared mailboxes didn’t have archiving enabled.

It’s important to note that Retention Hold is not the same as Litigation Hold – Litigation Hold puts a change freeze on a mailbox so that a user can’t delete or change items in their mailbox. It generally happens behind the scenes, and most users don’t notice that their mailbox has litigation hold applied, as deleted items disappear as normal, and the deletions/changes end up in a separate folder that the user cannot see.

Retention Hold, on the other hand, prevents the Managed Folder Assistant from running on that mailbox and processing retention tags. This means that users with Retention Hold enabled will not have their emails archiving or deleting based on the policies that have been set up in their Retention Policy.

After looking around a bit, I began to notice a pattern – each of the users who had their RetentionHold set to Enabled were users that we had been importing PST files into their online archives using the Office 365 Import Service. We had already noticed a bug (and opened a ticket), because we were not able to manually delete the jobs, and they weren’t automatically being deleted after 30 days the way they’re supposed to be.

It seems like Retention Hold is being enabled when a PST import job starts, and the flag is not being cleared automatically because the jobs are not being deleted properly.

You can fix this on a single mailbox by running the following command:

Set-Mailbox username@domain.com -RetentionHoldEnabled $false

Alternately, if you want to run this for all affected mailboxes, here’s the command to use:

Get-Mailbox -ResultSize unlimited | Where-Object {$_.RetentionHoldEnabled -eq $true} | Set-Mailbox -RetentionHoldEnabled $false

Hopefully Microsoft will get this bug resolved soon so that we have better control over the PST Import service – in the meantime, you can use this script to get retention policies functioning properly again.

Hope this helps!

Migrating to Office 365: The Good, The Bad, and The Ugly

Last night marked the launch of a new Cloud / Office 365 user group hosted by the team at Long View Systems – I had the pleasure of being the inaugural speaker, and we ended up having a great discussion around the various migration strategies available when moving email into Exchange Online.

It was awesome getting feedback from the folks that were at the meeting, and we had enough field experience in the room to be able to share our war stories, and talk about the good, bad, and ugly parts that make up the move into Microsoft’s cloud.

Hope you enjoy the show! 😀