Network Device Configuration Standardization – Thoughts on Ethan Banks’ post

Ethan Banks has an interesting newsletter called The Hot Aisle. Worth following if you’re not familiar with it, basically the thoughts of a very experienced network engineer.

In today’s post, Ethan covers the item at the top of his wishlist: network device configuration standardization. Ethan’s wishes are those of many others that I meet with on a regular basis. Many root-cause-analyses that were done over the years pointed to lack of config standardization as the root cause. Get that done well, and you get rid of many of the issues you run into regularly.

But how do you make sure it’s done well? In Ethan’s post he lists a few configurations he’d like standardized. I thought I’d add my $0.02 here on pitfalls people should watch out for. All of these I’ve learned through watching what issues our customers run into.

1. NTP server best practices – it’s one thing making sure all of your network devices uses the same NTP servers, it’s another to make sure they can actually reach them. Don’t just rely on tools that tell you the NTP config is set to the right host, test it! Different devices have different ways of achieving this.
2. External authentication – very important and common best practice. Be careful of a situation we’ve received multiple reports of (but found nothing online so far, interestingly): if you configure too many external authentication servers and all connectivity is lost, you might lose local authentication as well. The reason being that by the time the authentication times out with each external server, the entire login process times out.
3. SNMPv3 – good idea, just be careful of things like this.
4. SSH instead of Telnet – undoubtedly a good idea. We do see customers running older IOS’s that don’t support SSH.
5. Hardening – very important, but be very very careful about this. For example, hardening can result in GARP response packets being dropped, thereby breaking clustering of certain products. This means that you might not discover the impact of the hardening until months later.
6. Device configuration backup – VERY VERY important. Be sure to use a product that tests the backup was done correctly. You don’t want to try and restore a partial backup.
7. OOB management – one of those things people wish for when they’ve locked themselves out of a network (or a number of networks). The most common solution we see with customers are those console/lights-out products that use a separate DSL or cellular line to access. Expensive usually, much more than just setting up a VRF, but usually fool proof (if you actually monitor them to ensure they are connected).