Information Systems:Migration to externally-hosted DNS service

From uniWIKI
Revision as of 11:02, 24 April 2017 by Norwinu (talk | contribs)
Jump to navigation Jump to search 
This page will either remain a project page (in which case it should be marked to be archived), or will become a meta page for DNS configuration.

Overview

Project date: Mar. 2017

It has been decided that DNS hosting will be migrated from on-site hosting using Infoblox appliances to third-party DNS hosting providers. This configuration will be tried for a year, after which the solution will be reassessed.

DNS Configuration

External DNS Providers

There will be two hosting providers:

  • Primary: DNS Made Easy (username/password found here)
  • Secondary: Dyn.com (DynDNS) (username/password found here)
  • (Tertiary): On-site Infoblox devices (uni3sys.unipharm.com, uni4sys.unipharm.com)


Primary and Secondary DNS service:

It is important to note that service with Dyn.com is purely secondary DNS, and thus records cannot be managed there. Rather, Dyn servers will pull from DNS Made Easy servers through AXFR (an automated, background mechanism requiring configuration at both providers). Any changes made to DNS Made Easy servers will eventually be mirrored onto Dyn servers through periodic NOTIFY syncs, after which those records will begin to also be served by Dyn servers.

Name servers

DNS Made Easy
 ns10.dnsmadeeasy.com
 ns11.dnsmadeeasy.com
 ns12.dnsmadeeasy.com
 ns13.dnsmadeeasy.com
 ns14.dnsmadeeasy.com
 ns15.dnsmadeeasy.com

Dyn:
 ns2139.dns.dyn.com.	86400	Domain
 ns3199.dns.dyn.com.	86400	Domain
 ns1133.dns.dyn.com.	86400	Domain
 ns4155.dns.dyn.com.	86400	Domain

Round-robin nature of DNS queries:

Also of importance is the fact that with regards to DNS queries, there is no plausible way to influence name server priority i.e. which server gets queried first. Thus, DNS Made Easy is primary only in the sense that records are managed there. With regards to answering queries, all advertised servers (all NS record entries) have an equal chance of being queried and for the majority of the time are queried in round-robin fashion. They are all serving the same records, so this shouldn't matter, but it is important to keep in mind for the administrator. Also, for this reason, there should not be an entry for a server that is non-responsive (i.e. how uni4sys was configured with a Terago IP before), as it leads to a delay in DNS response time.

Hosted domains

The following are domains for which uniPHARM is authoritative:

  • unipharm.com
  • medicinecentre.com
  • umccharity.ca

(I guess that's actually it. Wow I thought there were more, but the other ones look defunct. -norwizzle (talk))

Billing

  • The plan at DNS Made Easy to which we are subscribed is Business Membership, allowing for 25 domains, 3 failover records, and 10 million queries per month (10 million per domain).
  • With Dyn.com, the service is Secondary DNS, where only a master IP is configured (the AXFR server at DNS Made Easy).
  • Secondary service is only enabled for unipharm.com and medicinecentre.com. Though we serve other domains, they are not as important as to justify a second level of redundancy through Dyn (the service cost is per domain).

On-site DNS: Infoblox Reconfiguration

Infoblox Trinzic 810 HA Pair Documentation (April 21, 2017) -darrenf

The HA pair of Infoblox appliances was wiped and returned to factory state in April of 2017. The setup was re-done by DarrenF so that the appliances could be used in a known good state before the support contract expired on May 8, 2017. A decision by management was made to not renew the support contract on the Infoblox appliances because 2 external DNS providers were put into place to provide redundancy to purely onsite DNS. Operating system updates are not possible after the support contract has ended. The NIOS version as of the end of April 2017 is 8.1.1. The very large 2000 page administration manual is on the Superserver in {unc | \\superserver.unipharm.local\tech\darrenf\Enterprise Applications\InfoBlox OS And Licenses |here} as well as the licenses and a copy of the OS file. There are 2 Trinzic 810 appliances that make up a high availability fail over pair. The devices were purchased in June of 2013 and installed in July 2013 by Gordian. The serial numbers and asset numbers are on the front panel of each appliance which is located in the Server Room.

The IP address configuration is below: 

uni3sys.unipharm.local		172.30.18.71
uni3sys.unipharm.com		207.102.139.98
uniblox.unipharm.local		172.30.18.71		This is a virtual name and IP that represents the Grid
uni1sys.unipharm.local		172.30.18.73            This is the LAN1 port IP of the top appliance
uni2sys.unipharm.local		172.30.18.75            This is the LAN1 port IP of the bottom appliance
uni1sysha.unipharm.local	172.30.18.72		HA port IP on uni1sys
uni2sysha.unipharm.local	172.30.18.74		HA port IP on uni2sys

Other details: 

*VLAN7 is used on the HA port connections on the Core Switch so that HA broadcasts don't leak onto the rest of the network
*SSH on port 22 is available to the .71 IP address to access the command line console
*Configuration backups of the Infoblox are FTPed to the Serv-U on the Mail server at 10PM each day using the "infoblox" account.  Files are saved in C:\FTP Root\Infoblox
*Infoblox appliances can send email alerts to webmaster@unipharm.com when services start or fail or when nodes reboot or fail

Use https://uni3sys.unipharm.com/ui to access the Infoblox web administration page. The username is admin and the password is NewVisionIT. It's important to understand that the HA pair is an active/passive setup where the virtual hostname and IP represent which ever physical appliance happens to be active. The appliance that is passive is just replicating whatever data changes are occurring on the active in the same way as the Sophos Firewall HA pair works. The HA port traffic is where the replication occurs and the LAN ports are there to link to the virtual IP. The public IP is NATted to the virtual IP at the firewall. The firewall allows queries from the outside public internet to flow to the Infoblox, but NIOS is only configured to answer queries for domains we own. The Infoblox will not answer a DNS query for other domains like www.apple.com or www.ibm.com, it will only answer for something.unipharm.com to mitigate DNSDDOS attacks. The Infoblox appliance(s) have their own highly customized version of BIND and our setup contains all the different domain names that uniPHARM owns such as medicinecentre.com and umccharity.com in addition to unipharm.com. Each domain "zone" has an SOA, NS, and any other needed A and MX records. The install of BIND on the Infoblox also has a reverse zone for our public IP addresses but as of the writing of this article, Telus still has the actual reverse PTR records. Each domain zone on the Infoblox is using NS records that include the DNSMadeEasy and Dyn services in addition to itself. If any changes are needed to that list of NS records, they need to be changed at the Grid level inside the Name Servers Group. It is always a best practice to change settings at the Grid level on the Infoblox so that both the Active and Passive nodes are configured the same way. NIOS allows Nodes to have different settings for flexibility in large and complicated environments, but having both Nodes use the same configuration is best for us.

Our Active Directory servers have the Windows DNS service installed on them and they are both configured to use the Infoblox as a forwarder in addition to using the Telus DNS servers as forwarders. This provides redundancy for queries going outbound from the Trusted LAN. If the Telus DNS servers are down (very unlikely) the Infoblox will use TLD root servers for queries. It is important to understand that the AD DNS servers naturally answer queries for unipharm.local, they also answer for unipharm.com. The key point here is that AD answers .com queries with what we want internal hosts to get instead of what a public internet host would get. For example, an internal building laptop querying for maggie.unipharm.com will be answered by AD with 172.30.18.15. A laptop at home using a home ISP can also query Maggie.unipharm.com, but it will be answered by the Infoblox instead of AD and the Infoblox will give the public IP instead of the private IP. Technically speaking the Infoblox can do this "split horizon" and they call it "DNS Views" but that greatly complicates its configuration. At the moment it is safer to separate AD DNS and public DNS as described above.

NTP service

The Infoblox provides other services besides DNS. The Network Time Service, or NTP has been configured and is syncing with top level NTP servers at the Canada Research Council. The AD servers get their time from the Infoblox and everything else (desktops, laptops, printers, switches) get their time from the AD servers. The firewall is configured to allow the Infoblox to connect to public time servers via TCP and UDP on port 123. Having all network objects configured with a consistent time source makes linking up events in logs on different systems easier, especially when trying to figure out difficult network failures. Please be aware that the NTP service on the Infoblox takes up to a day to properly sync if it is not able to connect outbound for some reason. The manually set internal clock on the active member of the HA pair must be within 60 seconds of whatever real NTP time is. Setting the manual clock requires the NTP service set to NOT synchronize AND a reboot occurs when the manual clock is adjusted. After the reboot, the NTP service may take up to a day to fully go green on the Infoblox dashboard.

TFTP service

The Trivial File Transfer Protocol is also setup on the Infoblox. It only allows connections from a range of IP addresses that are used by the network switches - 16.2 - 16.15. TFTP is used to save and upload the configuration files of the switches so that there is a backup. The 3com 4200G switches do not have the ability to save their configurations to an external TFTP server like the Infoblox, but the 3780 switches do.

Additional services

DHCP and IPAM are not turned on or setup on the Infoblox. There is a third hardware appliance - a model 250A that was setup by Gordian prior to 2012 and it serves DNS for the Terago connection. That connection and therefore the need to have DNS for it are scheduled to go away in very late 2017, early 2018. The 250A appliance is no longer supported by the manufacturer in any way and has an old and probably insecure version of NIOS installed on it. No changes to the 250A were made during the April 2017 re-do of the Trizic 810 pair. The use of the hardware appliances without any active support agreement is less than optimal but the risk of service outage to shareholders has been mitigated by the other external service providers. The risk of having the data or configuration on the Infoblox hacked is low but the risk increases if the appliances are accessible over the internet for many years without any OS updates. If a hardware failure occurs on one or both of the 810 appliances, there will not be any assistance from the manufacturer.

Additional notes

  • Setup of DNS Made Easy was simply through copying records on the Infoblox. A reassessment of the current records is recommended (e.g. orders.unipharm.com as a CNAME for bart.unipharm.com - why? -norwizzle)
  • Configuration TTL time is currently 1800 seconds (1 hour for all A records).
  • Failover records will not be added until multipath or active redundancy for the WAN is enabled.
Retrieved from "https://owl.unipharm.com/mediawiki/index.php?title=Information_Systems:Migration_to_externally-hosted_DNS_service&oldid=7429"