Reading

https://www.tecmint.com/install-ntp-server-in-centos/

https://linuxconfig.org/redhat-8-configure-ntp-server

https://access.redhat.com/documentation/en-us/red_hat_enterprise_linux/7/html/system_administrators_guide/ch-configuring_ntp_using_ntpd

Setting Up NTP Server - RHEL 7

1. Install and configure NTP daemon

yum install ntp

2. Edit NTP daemon configuration file and update with relevant external NTP source

# Edit /etc/ntp.conf
1. Comment the default list of public NTP server,
    update with required NTP source

    server  <ntp-source-1-fqdn | ip-address>  iburst
    server  <ntp-source-2-fqdn | ip-address>  iburst

2. Allow clients from the networks to synchronize time with this RHEL NTP server
    restrict <client-subnet> netmask <net-mask> nomodify notrap
        # nomodify notrap       
            # Clients are not allowed to configure the server or be used as peers for time sync
    restrict <client-subnet> mask <net-mask> nomodify notrap nopeer    # Alternative command

3. Add a log file statement which will record all NTP server issues
    # Used for troubleshooting
    logfile  /var/log/ntp.log

4. Adding a peer address
# The address of a NTP server of the same stratum
    peer <peer-ip-address>

3. Add firewall rules and start NTP daemon

firewall-cmd --add-service=ntp --permanent
firewall-cmd --reload

4. Start NTP daemon and enable it system wide

systemctl start ntpd
systemctl enable ntpd
systemctl status ntpd

5. Verify server time sync

ntpd -p
ntpd -R
ntpd -np    # Check NTP synchronization status
    -n  # Output all host address rather than host names
    -p  # Print a list of the peers known to the server as well as a summary of their status
ntpstat     # Obtain a brief status report

6. Query and synchronize against a pool of NTP sources

ntpdate  -q  <ntp-source-1-fqdn|ip-address>  <ntp-source-2-fqdn|ip-address>

# Verify date and time
    timedatectl

Setting Up NTP Server - RHEL 8

https://access.redhat.com/documentation/en-us/red_hat_enterprise_linux/8/html/configuring_basic_system_settings/using-chrony_configuring-basic-system-settings

https://access.redhat.com/solutions/58025

Recommended Configuration

It is NOT recommended to use only two NTP servers.

If more than one NTP server is required, four NTP servers is the recommended minimum. Four servers protect against one incorrect timesource, or "falseticker".

Best practices for NTP

https://access.redhat.com/solutions/778603

# Environment
a. Red Hat Enterprise Linux (RHEL)
    6
    7
    8
    9
b. ntp
c. chrony

General recommendations

1. Use at least 4 NTP servers

1. If you list just one, there can be no question which will be considered to be "right" or "wrong". But if that one goes down, you are toast.
2. With two, it is impossible to tell which one is better, because you don't have any other references to compare them with. This is actually the worst possible configuration. you'd be better off using just one upstream time server and letting the clocks run free if that upstream were to die or become unreachable.
3. With three servers, you have the minimum number of time sources needed to allow ntpd to detect if one time source is a "falseticker". However ntpd will then be in the position of choosing from the two remaining sources.This configuration provides no redundancy.
4. With at least four upstream servers, one (or more) can be a "falseticker", or just unreachable, and ntpd will have a sufficient number of sources to choose from.

2. Preferably use upstream NTP server for reference.
Default configuration (/etc/ntp.conf or /etc/chrony.conf) includes four pools that can be used:
    server 0.rhel.pool.ntp.org iburst
    server 1.rhel.pool.ntp.org iburst
    server 2.rhel.pool.ntp.org iburst
    server 3.rhel.pool.ntp.org iburst

3. Do not use a Virtual Server as NTP server.

NTP server was not designed to run inside of a virtual machine. It requires a high resolution system clock, with response times to clock interrupts that are serviced with a high level of accuracy. NTP client is ok to run in some virtualization solutions.

Implement RHEL 8 NTP chrony service

1. Install chrony NTP package

yum install chrony

Note:
a. The default location for the chrony daemon is /usr/sbin/chronyd
b. The command line utility will be installed to /usr/bin/chronyc.

2. Enable chrony to start after reboot

systemctl start chronyd
systemctl enable chronyd

# Check the status of chronyd
systemctl status chronyd

3. Set RHEL chronyd to act as an NTP server for the local network

# Update chrony configuration file /etc/chrony.conf
allow <client-subnet/mask>
    # Example
        allow 192.168.1.0/24

4. Restart chrony NTP daemon to apply the changes

systemctl restart chronyd

5. Open firewall port to allow for incoming NTP requests

firewall-cmd --add-service=ntp --permanent
firewall-cmd --reload

6. Confirm the RHEL NTP server configuration by manual time sync from the NTP source

ntpdate <ntp-source-fqdn | ip-address>
    ntpdate 192.168.10.1    # Example

7. Troubleshooting

# Checking if chrony is synchronized
chronyc tracking

# Checking chrony sources
chronyc sources

# The sourcestats command displays information about the drift rate and offset estimation
    process for each of the sources currently being examined by chronyd
chronyc sourcestats

Setting Up NTP clident on RHEL 8

Configure RHEL server as NTP client and sync time with required NTP server

1. Install chrony NTP package

yum install chrony

2. Enable chrony daemon to start after reboot

systemctl enable chronyd

3. Set chrony to act as an NTP client

# Update chrony configuration file /etc/chrony.conf
Server <NTP-server-fqdn | ip-address>

4. Restart NTP daemon to apply the changes

systemctl restart chronyd

5. check NTP server sources By default, the chrony NTP client will perform a time synchronization in every 64 seconds

# Run command on non-NTP server
chronyc sources

6. Check NTP client list on the NTP server

# Run the command on RHEL NTP server, to list of it ntp clients
chronyc clients

# Verify network ports
netstat -tulpn

Normally, the RHEL 8 server would have been configured as a NTP client. If this is the case, then simply set up an NTP server, and configure it to serve the network.

How to troubleshoot ntpd issues

https://access.redhat.com/solutions/64868

# Environment
    Red Hat Enterprise Linux 4
    Red Hat Enterprise Linux 5
    Red Hat Enterprise Linux 6
    Network time Protocol (NTP)

How ntpq works

# Run command
ntpq
ntpq> peers

 remote           refid          st  t when poll  reach  delay   offset   jitter
=====================================================================================
*time.rhl.       8.16.24.15       2  u  972 1024  377   28.066   -0.181   4.126
+dc1.riu.net     15.15.26.3       3  u  467 1024  377  141.664  -23.531   0.140
mighty.poclabs. .STEP.           16  u    - 1024    0    0.000    0.000   0.000
LOCAL(0)        .LOCL.           10  l  32   64   377    0.000    0.000   0.001

The "peers" command displays a list showing the DNS name or IP address for each association along with selected status and statistics variables.

The first character in each line is the tally code, which shows which associations are candidates to set the system clock and of these which one is the system peer.

During the clock selection process the refid will be .INIT. and the st (stratum) is set to 16.

a. The * indicates that this particular association is the chosen ntp source
b. The + indicates that this peer is a candidate
c. An empty space indicates that the peer unreachable and therefore rejected (stratum 16)

If no NTP servers get selected, get the output of the following commands for further debugging

ntpq> as

ind assID status  conf reach auth condition  last_event cnt
===========================================================
  1 29581  9624   yes   yes  none  sys.peer   reachable  1
  2 29582  9014   yes   yes  none  candidat   reachable  1
  4 29583  8000   yes   yes  none    reject

The "as" command displays a list of associations and association identifiers. Note the condition column, which reflects the tally code.

The associations shown above, correspond to the entries shown in the peer command. Use the "assID" for the following command:

ntpq> rv 29583      # rv assocID
    Note: 
    a. rv command displays the system variables billboard, including the system status word
    b. times are in milliseconds and frequencies are in parts-per-million (PPM)
    c. The server has successfully synchronized to its sources if the leap peer variable has value other than 3 (11b). 
        The client has successfully synchronized to the server when the leap system variable has value other than 3.
    d. A useful indicator of miscellaneous problems is the flash peer variable, which shows the result of 13 sanity tests.
        These bits should be zero for a valid server.
        https://www.eecis.udel.edu/~mills/ntp/html/decode.html#flash

Another useful aid is to run ntpdate with the -d switch

ntpdate -d <time-sourcce>
    ntpdate -d time.rhl.com     # Example

If there seems to be networking issue then tcpdump file port123.cap can be captured using below command for particular time duration of 20-25 minutes.

tcpdump -s0 port 123 -vvv -i <NIC> -w port123.cap