Firewall adventures: Transitioning from ISA 2006 to TMG

 

One of the key parts of my seemingly never-ending Offsite Replication project was to build out a second location to replicate my data to.  Before I could do this, some prep work to my network was in order.  It was a great opportunity for me to replace my existing firewall running Microsoft’s ISA 2006 server, to their newest edition, named ForeFront Threat Management Gateway, or TMG. 

My new TMG system is running on a 1u appliance provided by Celestix Networks, Inc.  Introduced to the Celestix line of appliances back in 2007, I’ve been very happy with the great turn-key solutions they provide.  Its great for those who want to run ISA/TMG, but do not want to build up their own unit, and do not want to handle licensing of the OS or TMG.  The lineup they offer ranges anywhere from branch office solutions to backbone class systems  Some really nice abilities are built right into the unit, such as web based management, and updating the unit to a new build by booting to PXE.  It also offers a “Last Good Version” (LGV) that will reimage the disk the the state it was saved, in the event of a configuration change going terribly wrong.  Definitely peace of mind for those critical upgrades.  The nature of the image creation and restore is such that it requires the system to be offline.  I hope that in the future, Celestix can perhaps partner with Acronis, or some other disk imaging solution to make this process a little more convenient.  It still works pretty well though.  Anyway, onto the transition.

 

Upgrade, or transition?

This seems to be one of those ubiquitous IT related questions to almost any enterprise solution that is being run in a production environment.  Should you do an in place upgrade, or should you transition to a pristine installation?  In this particular case, this was already answered for me, as my old appliance ran a 32bit version of Windows Server 2003, and could not be upgraded due to system requirements.  That was okay with me.  A true upgrade fell out of favor with me years ago; there are just too many unknowns introduced, which can make post deployment issues extremely difficult to diagnose.  I’ve also sensed that the true upgrade has fallen out of favor with software manufacturers as well.  Whether it’s Exchange, SQL, or a server OS, the recommended way these days seems to be transitioning to a pristine installation.

 

The new box

For the new environment I was building, I chose two Celestix MSA5200i units; one for the primary facility, and one for the CoLocation.  These particular units run TMG Standard, on top of Windows Server 2008R2. It would have been nice to go with a unit running the Enterprise Edition of TMG (that offers the ability to create a redundant array of servers), but I had to cut costs, and going with the Standard Edition was the easiest way to do this.

With the new unit sitting in front of me, I decided to build it up in its entirety offline, and wait for a weekend to cut it over.  ISA has the ability to dump out all, or parts of the old configuration in XML, so my early (albeit naive) visions had me thinking that my transition steps would simply be exporting the configuration running on the ISA 2006 box, and importing it to the TMG box.  Well, the devil is in the details, and while this could work for certain scenarios, it didn’t work for me on the first a few tries.  I had a choice.  Continue chasing down the reason why it wasn’t importing (an unknown time limit), or pound out a new configuration in a few days (a known time limit).  No time to complain – just do it and get it over with.  Good documentation in OneNote, and the ability to RDP into your existing ISA installation is key to this being a successful way to build a new configuration from scratch.  To minimize typos and other fat fingering, I did export custom sets and protocols at the very granular level.  Sure, I could type them out easy enough, but it was more reliable to export at the very small item level.

A properly configured TMG box is almost always joined to Active Directory, and there are some steps that you just have to wait to get to on the day of transition.  This is reasonable, but it does have to be planned for.  Things like using Kerberos Constrained Delegation in publishing rules, can only be configured after it’s joined.  It’s also worth making sure you know all AD related settings (Delegation, OU location, GPO overrides, etc.) for the existing Firewall that you will be decommissioning.  Nothing like a oversight here to mess you up.

 

Post installation surprises

The abilities of TMG make it far more than a simple edge security device.  It is what truly separates it from the competition.  Since it is integrated into the operation of so many functions up and down the protocol stack, transition like this can be a bit disruptive.  I’m happy to say that considering the type of change, I didn’t run into too many troubles. I had prepared a checklist of basic functions and services I could run over to quickly validate a successful transition.  This made validation easy, and prevented most Monday morning surprises. 

After about 20 minutes, I had the old ISA box removed from the domain, and the new one added and configured.  The rest of the time was spent confirming functionality, and resolving a few issues.  Here were some of the minor ones:

• ARP caching.  This isn’t the first time this has bitten me.  I forgot that the ARP cache on the connecting devices needed to be flushed.  Silly mistake, but the nice part is, that it eventually corrects itself. (I wish I had a few more of those kinds of problems).
• Publishing rules and Listeners.  After you join the box to the domain, you will want to check these, and recreate if necessary.  I had a few publishing rules that I had to recreate.  Not a big deal.  They looked okay, but just didn’t work.
• I have several publicly registered IP addresses bound to the external (WAN) interface.  Windows  2008 and TMG didn’t bind to the IP address I was thinking it was going to bind to (or at least the way Win2003 and ISA did).  A quick fix in the TMG configuration resolved this.   Look to this TechNet Article on why the behavior is different.

The final issue was a little trickier to fix.  The symptoms were that web browsing was working, but it just took a while to connect.  After looking at the logging, (and being tipped off on a thread on isaserver.org’s community forum), I noticed that the web proxy was attempting to use one of the RRAS adapters as the default gateway.  It was being caused by web proxy clients getting confused when reading WPAD for automatic browser/proxy configuration.  The slow browsing would go away as soon as the web browser’s proxy settings were manually configured.  Apparently this behavior wasn’t unique to TMG (others on ISA 2006 have experienced similar behavior), but this was the first time I’ve ever seen it. 

There was a .vbs script that supposedly fixed the issue.  The purpose of the .vbs script was to insert the FQDN of the TMG unit into WPAD.  While the script ran successfully, it didn’t change the behavior for me.  At this point, a little bit of panic set in.  I thought it best to tap into the expertise of my good friend, and TMG superstar Richard Hicks.  Richard is a Microsoft MVP, and has a great blog that should be in everyone’s RSS feed list.  After briefing him on the scenario, he provided me with another script (courtesy of Technet) that would attempt to achieve the same results as the failed script.

‘http://blogs.technet.com/isablog/archive/2008/06/26/understanding-by-design-behavior-of-isa-server-2006-using-kerberos-authentication-for-web-proxy-requests-on-isa-server-2006-with-nlb.aspx

Option Explicit

Const fpcCarpNameSystem_DNS = 0
Const fpcCarpNameSystem_WINS = 1
Const fpcCarpNameSystem_IP = 2

Dim Root, Array, WebProxy

Set Root = CreateObject("FPC.Root")
Set Array = Root.GetContainingArray
Set WebProxy = Array.ArrayPolicy.WebProxy

If fpcCarpNameSystem_DNS = WebProxy.CarpNameSystem Then

  MsgBox "ISA is already configured to provide DNS names in the WPAD script.", vbInformation
  WScript.Quit

End If

WebProxy.CarpNameSystem = fpcCarpNameSystem_DNS
WebProxy.Save true

MsgBox "ISA was configured to provide DNS names in the WPAD script.", vbInformation

Set WebProxy = Nothing
Set Array = Nothing
Set Root = Nothing

After I applied the .vbs script above, the issue has seemed to resolve itself, and now it’s all running smooth. 

 

Observations

During my initial build of the new TMG unit, the first thing I noticed was the apparent efforts the TMG Team took to maintain the same look and feel as the previous version.  I had seen screenshots of TMG, but that doesn’t give a good feel for UI interaction.  Aside from the new features, it was quiet a relief to feel instantly comfortable with the UI.  What a welcome relief to the overworked IT guy.

The next step was to give myself a refresher on what was new with TMG, and digest how that was going to influence my configuration after the cutover was complete.  The improvements really do read like a wish list for the seasoned ISA 2006 user.  Sometimes the Value Proposition for a software manufacturer, and their customers don’t match up.  The result is this odd rollout of new features that the customer never asked for, and ignoring what the customer wants.  That doesn’t seem to be the case at all with this product. 

For my transition, it was most prudent for me to delay taking advantage of some of these features, just to reduce all variables, but will definitely be exploring the great features of of TMG in the coming weeks and months.  The top priority right now is getting my second TMG unit built and configured for my CoLo facility, and test my replication.  That’s what a deadline does for you.  It ruins all the fun.

Once again, a big thanks to ISAserver.org for being a great resource for the ISA/TMG user community, as well as the folks at Microsoft, Rich, and the others at Celestix for making a quality product.