DNS as a Single Point of Failure

By Justin HittSeptember 7

Your business website profits depends on a reliable infrastructure, including a fast performing domain name services (DNS). Unfortunately for most business websites, DNS is their biggest point of failure.

Domain name services a.k.a. DNS is how website names become numeric addresses. Like directory assistance, you ask for a name and DNS gives you an address. DNS, it's how you find stuff on the Internet.

In the 90's I was building out a network of sites for a client. They got good traffic and my clients focus was all about hosting. They wanted the fastest servers, the most amount of ram, and big disks.
As the results of a platform risk assessment, I pointed to a single DNS server they managed (this was before big hosting) that contained the only copy of all their domain names. This server contained everything about all of their websites, yet wasn't backed up, nor did it have a secondary server for fail-over.
Excited about their growth, this client ignored my advice. As they grew to more than 1,000,000 impressions a day, all focus was on load balancing their web hosting servers. They pick up more traffic with media coverage and poor ole DNS was ignored.
One day someone turns off the wrong box, the DNS server, and all that hard work halts. Like a heart attack traffic stops cold. It doesn't matter what kind of load balancing or how fast your server is if your visitors can't resolve the name of your server.
Fast forward to 2007, working with a high volume Internet publishing company with more than 4,000 DNS name resolves a second. They have two Linux Bind9 DNS servers. The systems manager has the forethought to put in several DNS appliances, plus load balancing for DNS, and establishes a name management policy.
Due to the volume of traffic, bad records, even gremlins, this publisher could lose half their DNS servers before any noticeable change in performance. They can also sustain the requests of the best marketing campaigns that spike traffic (which they have on a monthly basis.)
Even your hosting provider may not protect your DNS address records.
Recently I had a conversation with the owner of a medium-sized hosting provider. While they had a redundant DNS infrastructure, it wasn't working properly making DNS a single point of failure. He tells me an outage would cost his company tens of thousands dollars each hour of downtime.
Whether you are a start-up or a large publishing company, many business websites are not adequately protected against DNS failures, is yours? Failures include poor server configuration, bad records, DNS cache poisoning or even malicious code from outside.
There are a number of things you can do depending on budget and available resources. Yes, you may need to spend a little money, unless you are working with a qualified systems administrator with DNS experience (even then you may need equipment.)
Right now you need to make a backup of your DNS zone records, then work with your systems team or a qualified contractors to flesh out a plan. If you are working with a webmaster service provider, some are qualified to support you in this effort.
You would benefit from diagramming your existing environment noting any dependencies. Note how machines talk with each other, domain forwarders, secondary name services, and anything depending on this environment. Making DNS redundant is critical to the long term stability of your business website.
How you reorient to make your DNS infrastructure redundant depending primarily on the type of software you are using. For my web hosting pools I'm using DNS Made Easy‘s Anycast, however, for volumes over 1,000,000 resolves a day you may need a different solutions.
If you are using Infoblox Appliances, have F5 BigIp LTM/GTM services, or just Bind9 (which comes with most Unix machines), then each would have a different approach for distribution. In some environments I've used a combination of all these platforms for a low cost DNS infrastructure that was easy to grow as needed.
Fortunately, Correcting the problem costs much less than you'll lose with an outage. In another articles I'll cover tests to determine if your DNS is a single point of failure and to measure your vulnerability. What's your DNS plan?
© 2009 B2B Website Profits, All rights reserved.

Justin Hitt

About Justin Hitt

A business analyst who specializing in sales generating marketing copy. Author, copywriter, and publisher of newsletters that help clients transform business relationships into profits guaranteed. Reach him by Fax at +1 (877) 486-8461

{"email":"Email address invalid","url":"Website address invalid","required":"Required field missing"}