September 2011 IT Business Consulting Newsletter

Hurricane Preparation and Remote Operations

By Tom K

What you need to do in preparation of a serious storm depends on the severity of the storm, the type of storm expected, and the vulnerabilities of your site.

Typically, we are concerned with storm surge entering the building, structural damage from winds or flying debris, windswept rain entering through breached windows, walls, or roof, and electrical damage from lightning or “shorts”.

Once your local IT resources are protected to the best of our ability, we need to consider ongoing operations while the effects of the storm impact your environment.

In this month’s newsletter I’ll present multiple recommendations that have helped my clients mitigate IT loses and maintain operations through a number of severe hurricanes. I hope you find it helpful as well.


Servers and Backup Tapes

While the protection methods vary for the four concerns, if you do fear damage at your site, you should move your backup tapes and your key server(s) to a very safe place. You might want to consider moving each to a different safe place. These are typically just temporary storage locations within your facility, but may be off-site depending on conditions.

If you need to evacuate and you use an SBS server, I suggest you take it with you (or with someone going inland), and send the BU tapes inland with multiple staff (one daily tape each). You should have a backup strategy in place that includes daily backups. See Backup the Company Jewels! for a thorough discussion of Backups.

It is also a good idea to put your critical data on several Memory Sticks / Thumb Drives and distribute them to key personnel. They are cheap, and a 16 GB drive can hold a lot of data (PMS databases + financials + more).

If you can easily remove any secondary servers, you should. If not a priority, treat them like a PC as described below.


Electrical Damage Protection

You should be concerned with electrical storm damage and electrical damage from internal electrical faults caused by the storm, so unplug all PCs, Monitors, and Printers, and disconnect the Ethernet (LAN) cables as well. You should also disconnect the cable and DSL connections from the modems, and unplug these devices along with the firewall and your Ethernet switches. You may also want to shut down and unplug your UPS’. Unplug FAX machines, and disconnect their phone and Ethernet connections.

Ultimately, you don’t want any cords plugged into any device – Ethernet, phone, cable, DSL, or power.

Consult with your phone switch provider to determine what is recommended for its protection.


Storm Surge

This is pretty simple. Get everything you want protected off the floor and as high up as possible. This usually includes PCs, UPS’, power strips, and small Ethernet switches. Most surge situations we have seen didn’t get higher than the desktops, but this is dependent on how close you are to the sea, and how high the ground is at your location.

If you have an inside room (windowless) on the second floor, this would be a good place to store your PCs and electronics.

If you can move your PCs to a very safe location off site, you may do so, but most companies don’t do this. PCs are not on most peoples’ evacuation lists.

Consider your infrastructure components, especially if they are located on the first floor, or near a window. If you lose your main Ethernet switch, your whole operation will be down until it is replaced. If you lose a modem or firewall, you’ll have no Internet until it is replaced.


Rainwater and Debris

This is also pretty simple. If conditions are such that this is a concern for your site, move everything you want protected into the safest place possible – an internal (no windows) office or storeroom is best. You can also put the devices in large garbage bags, and put them on tables or desktops, again in the internal rooms if you have any.


eMail

If you outsource your email, you’re all set.

If you run your own on-site mail server and subscribe to an off-site spam filter, these devices will usually hold your email in queue for at least two days. If the techs managing these devices are alerted to the need, they can usually increase the queue time, sometimes to several days depending on the device. You should call your provider’s tech support to enquire of the max queue time & advise them you may need it.

If your mail comes directly to your email server, when you disconnect your server any mail server trying to send mail to yours will continue to try for 1 – 2 days. If the sending server can’t reach your server within its time-out period, it will send a non-delivery report to the sender.


Web Operations

If you use Property Plus and one of the Instant Software links, your web site will present property pages as usual while your server is disconnected, but the web site can not normally do real-time availability and booking without being connected to your data server, as your data server has ownership of this data.

If you use Property Plus with Local Social, their disaster process is in play for their potentially affected clients. Your Property Plus will be live on a remote virtual machine that will communicate with your web site to maintain on-line bookings, and you’ll be provided with remote administrative access to your interim Property Plus server. If the impending storm turns into a disaster, access can be stepped up so general staff can work within your interim remote Property Plus for an extended period of time. This remote virtual server does have some additional disk capacity, and you are free to use your LSI provided virtual server for non-PMS data and applications as you see fit.

If you use Property Plus with other web providers, you may want to give them a call to see if they have any systems in place to handle transactions should your data server be off-line for any length of time.

If you use a web based PMS like Escapia, Barefoot, V12, or VRM, you're all set.


Remote Operations

If you are using an SBS server for your business operations and you need to evacuate, or choose to temporarily shift your operating location, it is a relatively simple matter to pick up what you need and relocate anywhere there is a broadband Internet connection.

If your infrastructure was set up properly, all you’ll need is your SBS server, your main site firewall, and a PC/laptop. Nice to have’s are a UPS and a printer. If you need more than 2 PCs/laptops, bring an Ethernet switch.

If you have multiple servers, the process is the same and relocation is very doable, we’d just need to move a few more servers (after selecting which we really need) and an Ethernet switch.

Once you are physically set up in the temporary location, we’d need to edit your firewall configuration. I can walk you through the initial steps to get it to where I can access it from outside. We would then have to advise some of your other service providers of your temporary public IP address, so they could adjust their data and systems.

Your phone provider should be able to easily route your business calls to the temporary site’s local phone number, or your cell.

For future consideration...

We have set up evacuation sites for some of our clients who operate in storm areas. These sites are typically inland homes of friends, relatives, or colleagues. The investment is minimal, and the sites are pre-configured so the Property Managers can drop in their SBS servers and go live in minutes.

We install a firewall ($130), an Ethernet switch ($150) if needed, sometimes a UPS ($450), and we duplicate the internal office IT configuration on the firewall. We get a business Internet subscription with a static IP address. We document the public address info and send it to the service providers who need it, so they can easily make all necessary changes when we call and say “moving to site B in twenty minutes”. There is not much work to set it up, very little expense, and it is always ready for you.

When we are not using the evacuation site (and we hope to never use them), the residents have a typical home network, except it has a static IP. When we need the site, we simply plug in the SBS Server & make a few quick phone calls, and we are fully operational.


If you have any questions about any of this info, or if there is anything I can do for you to help you prepare, storm prep has absolute priority. Please don’t hesitate to contact me at TomK@TomKConsulting.com, or via my cell 443.310.5110.


Next month I’ll discuss the concerns IT should have relating to departing employees, the steps we recommend to properly deal with security and auditing, and different processes used when the departing employee is leaving with blessings or in handcuffs. See "Departing Employee? How to Process them Gracefully and Securely".