Published on

Microsoft Exchange 2016 and 2019

Authors
  • Name
    Jackson Chen

Exchange Server 2019

Supported coexistence scenarios for Exchange 2019

The supported coexistence scenarios between Exchange 2019 and earlier versions of Exchange are described in the following table:

Exchange version                    Exchange 2019 organization coexistence
Exchange 2010 and earlier versions  Not supported
Exchange 2013                       Supported with Exchange 2013 Cumulative Update 21 (CU21) or later on all Exchange 2013 servers in the organization, including Edge Transport servers.
Exchange 2016                       Supported with Exchange 2016 CU11 or later on all Exchange 2016 servers in the organization, including Edge Transport servers.
Mixed Exchange 2013 and Exchange 2016 organization  Supported if all Exchange 2013 and Exchange 2016 servers in the organization meet the requirements as previously described in this table.

Exchange Server 2016

Exchange Server 2013

Configuration, Roles and Services

Exchange Active Manager and Primary Active Manager (PAM)

Microsoft Exchange Server includes a component called Active Manager that manages the high availability platform that includes the database availability group (DAG) and mailbox database copies. Active Manager runs inside the Microsoft Exchange Replication service (MSExchangeRepl.exe) on all Mailbox servers. On Mailbox servers that aren't members of a DAG, there is a single Active Manager role: Standalone Active Manager.

On servers that are members of a DAG, there are two Active Manager roles: Primary Active Manager (PAM) and Standby Active Manager (SAM). PAM is the Active Manager role in a DAG that decides which copies will be active and passive. PAM is responsible for getting topology change notifications and reacting to server failures. The DAG member that holds the PAM role is always the member that currently owns the cluster quorum resource (default cluster group). If the server that owns the cluster quorum resource fails, the PAM role automatically moves to a surviving server that takes ownership of the cluster quorum resource. In addition, if you need to take the server that hosts the cluster quorum resource offline for maintenance or an upgrade, you must first move the PAM to another server in the DAG. The PAM controls all movement of the active designations between a database's copies. (Only one copy can be active at any specified time, and that copy may be mounted or dismounted.) The PAM also performs the functions of the SAM role on the local system (detecting local database and local Information Store failures).

The SAM provides information on which server hosts the active copy of a mailbox database to other components of Exchange that are running an Active Manager client component (for example, Client Access or Transport services). The SAM detects failures of local databases and the local Information Store. It reacts to failures by asking the PAM to initiate a failover (if the database is replicated). A SAM doesn't determine the target of failover, nor does it update a database's location state in the PAM. It will access the active database copy location state to answer queries for the active copy of the database that it receives.

Note:
1. The DAG member that holds the PAM role is always the member that currently owns the cluster quorum resource (default cluster group). 
2. If the server that owns the cluster quorum resource fails, 
    the PAM role automatically moves to a surviving server that takes ownership of the cluster quorum resource. 
3. If you need to take the server that hosts the cluster quorum resource offline for maintenance or an upgrade, 
    you must first move the PAM to another server in the DAG.
    Note:
        Only when taking the PAM server offline, then move the PAM role to another server.

Note:
    Each DAG member that does not own the Cluster Group is a Standby Active Manager (SAM)
# How to verify Primay Active Manager
Note: The PAM role is held by whichever server holds quorum

Get-DatabaseAvailabilityGroup -status | FL
Get-DatabaseAvailabilityGroup <DAG-Name> -status | FL Name, PrimaryActiveManager

# Run this command, and it will shows the server which owns the cluster group, which is the PAM
Cluster Group
Get-ClusterGroup    # list the state and owner of each cluster ndoe, or resource group in local cluster     


# How to move PAM to another server
Get-ClusterGroup “Cluster Group” | Move-ClusterGroup –Node <Other DAG Member>
Move-ClusterGroup -Name <cluster-group-name> -Node <node-name>
        # where <node-name> is the new owner of PAM


# Force move PAM (offline) to another node
cluster.exe <DAG-Name> /MoveTo:<new-node>   # new node is where PAM will be moved to

Troubleshooting

How to move mailbox datatabase Active server to another Healthy DAG member server
# Check and ensure there is a healthy mailbox database in DAG member
Move-ActiveMailboxDatabase <database-name> `
    -ActiveOnServer <Healthy-DAG-Node-Name> `
    -SkiphealthChecks -SkipActiveCopyChecks SkipClientExperienceChecks -SkipLagChecks -MountDialOverride:BESTEFFORT

Once the database moves to the healthy DAG member and it will be able to be mouted automatically, the clients will be able to connect to their mailboxes.

How to remove failed DAG member server
# Identiy the status information of all copies of the required database
Get-MailboxDatabaseCopyStatus -Identity <mailbox-db-name> | FL
    # It will show status, such as Healthy, Failed

# Check mailbox database copy status on mailbox database server
Get-MailboxDatabaseCopyStatus -Server <server-name> | Format-List

# Identify the failed DAG member, and remove the copy of DAG mailbox database
Remove-MailboxDatabaseCopy -Identity <mailbox-database-name>\<failed-data-node> -Confirm:$False

# After successfully remove the copy of the DAG mailbox database
Remove-DatabaseAvailabilityGroupServer -Identity <DAG-name> -MailboxServer <failed-data-node> -ConfigurationOnly
Monitor database available group

https://docs.microsoft.com/en-us/exchange/high-availability/manage-ha/monitor-dags?view=exchserver-2019

# View status information about mailbox database copies
Get-MailboxDatabaseCopyStatus

# View continous replication status
Test-ReplicationHealth
Test-ReplicationHealth -Identity <mailbox-server-name>
CollectOverMetrics.ps1

Exchange Server includes a script called CollectOverMetrics.ps1, which can be found in the Scripts folder. CollectOverMetrics.ps1 reads DAG member event logs to gather information about database operations (such as database mounts, moves, and failovers) over a specific time period.

CollectOverMetrics.ps1 -DatabaseAvailabilityGroup <DAG-Name> -Database:"DB*" -GenerateHTMLReport -ShowHTMLReport
CollectReplicationMetrics.ps1 script

CollectReplicationMetrics.ps1 is another health metric script included in Exchange Server. This script provides an active form of monitoring because it collects metrics in real time, while the script is running. CollectReplicationMetrics.ps1 collects data from performance counters related to database replication. The script gathers counter data from multiple Mailbox servers, writes each server's data to a .csv file, and then reports various statistics across all of this data (for example, the amount of time each copy was failed or suspended, the average copy or replay queue length, or the amount of time that copies were outside of their failover criteria).

# Example gathers one hour's worth of data from all the servers
CollectReplicationMetrics.ps1 -DagName <DAG-name> -Duration "01:00:00" -Frequency "00:01:00" -ReportPath

Update mailbox database copy

https://docs.microsoft.com/en-us/exchange/high-availability/manage-ha/update-db-copies?view=exchserver-2019

https://docs.microsoft.com/en-us/powershell/module/exchange/update-mailboxdatabasecopy?view=exchange-ps

Updating, also known as seeding, is the process in which a copy of a mailbox database is added to another Mailbox server in a database availability group (DAG). The newly added copy becomes the baseline database for the passive copy into which log files copied from the active copy are replayed. Seeding is required under the following conditions:

  1. When a new passive copy of a database is created. Seeding can be postponed for a new mailbox database copy, but eventually each passive database copy must be seeded to function as a redundant database copy.
  2. After a failover occurs in which data is lost as a result of the passive database copy having become diverged and unrecoverable.
  3. When the system has detected a corrupted log file that can't be replayed into the passive copy of the database.
  4. After an offline defragmentation of any copy of the database occurs.
  5. After the log generation sequence for the database has been reset back to 1.
# How to seed a copy of the database DB1 on MBX1 using MBX2 as the source Mailbox server for the seed.
Update-MailboxDatabaseCopy -Identity DB1\MBX1 -SourceServer <Active-Database-Server>
    # Need to use the Exchange server name which has the database status as ACTIVE

    Update-MailboxDatabaseCopy -Identity DB1\MBX1   
        # without specify source server it  will use the active mailbox database server

# Verify the successful seed status
Get-MailboxDatabaseCopy <databasename>   # where databasename is the seeding database in the passive server

Reseeding after ‘Mailbox Database Copy Failed & Suspended’

https://www.nucleustechnologies.com/blog/update-mailboxdatabasecopy-failed-and-suspended/

Things to consider
  1. The reseeding process will take time as per the size of the database and the network speed between the source and destination Exchange Servers.
  2. By default, the cmdlet will use only the DAG group member server with the active database copy as the host server to copy the database.
  3. The final data at the destination will be larger than the source database. For example, if the size of the database is 100 Gigabyte, the database at the destination will have more than 100 Gigabyte because it will consist of transaction log files and content index information additionally.
  4. If the source and destination Exchange Server belongs to the same DAG group, the copy process will take less time. If the servers belong to different DAG groups, the copy process may take significant time to complete.
# The situations in which seeding/reseeding is beneficial
1. The original database is unavailable after a server crash
2. The Exchange log files are corrupt and cannot be played on the original database
3. The original database hasn’t undergone offline defragmentation

#***** Preparation
You need to suspend the database copy before running the update cmdlet.
https://docs.microsoft.com/en-us/exchange/high-availability/manage-ha/suspend-resume-db-copies?view=exchserver-2019

# suspends continuous replication for a copy of the database DB1 hosted on the server MBX1
    Suspend-MailboxDatabaseCopy -Identity DB1\MBX1 -SuspendComment "Maintenance on MBX1" -Confirm:$False

# Seeding/Update databasecopy
Update-MailboxDatabaseCopy -Identity <failed/Suspended-DB>\<EX-ServerName> -SourceServer <Healthy-Active-DB>
Update-MailboxDatabaseCopy -Identity <failed/Suspended-DB>\<EX-ServerName>

# Check copy status
Get-MailboxDatabaseCopyStatus
Get-MailboxDatabaseCopyStatus <DatabaseCopyName> | Format-List

# Other commands to fix the issue
a. To fix the error, you need to update the mailbox database copy running the cmdlet:
    Update-MailboxDatabaseCopy -Identity “DB322312\NDR32” -DeleteExistingFiles
b. For longer reseeds, if you want to keep the PowerShell closed, then use the below cmdlet:
    Update-MailboxDatabaseCopy -Identity ‘DB322312\NDR32’ -BeginSeed
c. The DeleteExistingFiles cmdlet deletes the existing log files to avoid any error messages:
    Update-MailboxDatabaseCopy -Identity “DB322312\NDR32” -BeginSeed -DeleteExistingFiles