Network management tools

Today’s Web is more complex, and its underlying infrastructure is more involved. Web software now comes with more options and choices. Sites now are all about retaining visitors with value-added services and improving network latency and site reliability. This is a very different situation from the good old days of the Web (say 1995), where most Webmasters and site operators were concerned about putting together good-looking HTML pages or offering ways to capture a visitor’s attention with animated graphics and interesting backgrounds.

The recent sophistication of the Web is both a blessing and a curse. New technologies such as caching, load balancing, and network monitoring and management will take some time and effort to figure out. It also will take time to understand what works best for your particular situation. For example, if you are running a site with multiple Web servers located in different locations, your management needs will be more challenging than if your data center is under one roof. That’s where this series, “New Web technologies to improve your site,” will hopefully help you stay on top of these new technologies and decide which ones are most appropriate for your own Web site.

This article is the first of four practical surveys of new Web technologies that you can use to enhance your site. Over the next few months, I’ll cover what is new and different about the Web as we approach the year 2000. This article will review tools to help you manage your Web site and keep track of its reliability using some very easy and inexpensive services and products. Part 2 will look at the universe of Web caching, including the various kinds of services and server technologies that can be used to improve the Web browsing experience both for your own network and for your visitors from around the globe. Part 3 will cover new kinds of payment processing technologies that are being put in place for a number of leading Web storefronts. Finally, Part 4 will examine the growing variety of outsourced services that you can use to enhance your own Web site and add depth in terms of e-commerce, hosting management, and additional services such as customer response management and e-mail outsourcing. Each article will provide examples of leading-edge technologies and links, whenever possible, to where you can try limited-term downloads of software.

I assume that you have some familiarity with installing and operating your own Web server, whether this is on Windows NT, UNIX, or some other operating system. While many of the tools and servers are only available for NT, many will also work on other platforms, or are available directly from Internet-based services and will work with any Web server. With any product, I will go into details about my usage and experiences, both good and bad, based on years of operating several Web sites and testing hundreds of different products.

Site monitoring

The world of Web server management takes in a fairly wide swatch of technologies. It also helps you–as a Webmaster or a CIO–accomplish several different tasks:

• Ensuring that all aspects of your site are operating 24/7, or as close to continuous operations available to the Internet as you can afford.
• Knowing what others, such as your competitors or allies, are saying about you, your products, and your corporation across the public Internet.
• Ensuring that your site can be viewed by the widest possible collection of browsers and environments.
• Ensuring that your site can deliver consistent performance, especially during peak load periods.

In order to accomplish these tasks, you’ll need several different tools and you’ll need to examine the various components of your Web site carefully. First, your Web server and other associated servers such as database, directory, and e-mail servers, all need to be up and available to the Internet. This means monitoring the various services or programs that are running on these servers and making sure that none has crashed or terminated unexpectedly.This also means understanding your overall Web content and how this content depends on each of these services. For example, your Web storefront’s catalog may depend on a database server to display current items and inventory. Or you may have a form that will e-mail Web site visitors a confirmation message that depends on e-mail services working properly, as well as on the program that processes the underlying scripting language that creates the form. You might need to be able to connect to your corporate directory to route an inquiry to the appropriate mailbox for follow-up. And your Web site may make use of several other tools and processes, all of which require their own set of services to be operating, such as Allaire Corp.’s ColdFusion or database middleware tools.

There are several tools to help with these monitoring tasks (see table, “Web management technologies“). If you are running NT servers, perhaps the best product is App Manager Suite from NetIQ Corp. This software product contains three components: First are a series of software agents to monitor the particular service of interest, whether this is an Exchange server, SQL Server, IIS, or general NT services. Second is a management console that keeps track of the agent and the service and sends alerts. Finally are additional modules to communicate with the alert system, ranging from alphanumeric pagers to more sophisticated network management frameworks such as Computer Associates Inc.’s Unicenter TNG or Hewlett-Packard Co.’s OpenView.

Additional products include Sitescope from Freshwater Software Inc., which has similar monitoring components but also runs on a few different UNIX platforms in addition to NT. A different sort of product is ServerSitter from FastFocus Systems Inc. This takes the form of a special add-in adapter card that sits inside your NT server and monitors it constantly to ensure that it is running reliably. When the card detects the server has crashed or certain critical services have failed, it sends a signal to reboot the machine. Given that NT can be unstable under certain circumstances, an investment in this product seems prudent.

Site connections

But these products just deal with monitoring your server. Another part of the overall network management equation is dealing with your physical connection to your Internet Service Provider (ISP). This includes your own internal network connections as well as the links and routers to the public Internet through your ISP. These connections can include the equipment, such as a T1, ISDN, or digital subscriber line, that connects your company to the Internet, as well as the routers, hubs, and firewalls used as part of maintaining this connection. This often means providing for back-up power and redundant network connections in case one of these components fails, as well as keeping track of when your ISP link is down or unreachable.

There are several services that use the Internet itself to keep track of your connection to it. The ones I have used include WebPartner from VerticalSoftware.com Inc. and Phil Greenspun’s Uptime, both freely available. The way they work is, you first go to their respective Web sites and sign up for the service, which normally takes just a few minutes. You provide the URLs that you wish to keep track of and an e-mail address for the alerts, along with some other information such as how often to check and how often to send you scheduled summary reports, usually on a daily or weekly basis. When the URL isn’t reachable for several minutes, an alert is sent to your e-mail address notifying you of the problem. WebPartner requires no additional changes to your Web content, while Uptime requires a single, simple text file to be uploaded to your Web site, which it uses to locate your site. This file will take most Webmasters just a few minutes to prepare, though.

A more continuous service is WhatsUp from Ipswitch Inc., which is a piece of Windows software that does something similar to the two service offerings mentioned above, and can be tailored to send out alerts via a wide variety of mechanisms, including pagers and e-mail. And there is Net Resolve’s Site Watch 2000, which has a variety of service offerings to monitor your site, including the ability to report back on overall network latency from a number of places around the Internet.

If you like the tools available from Freshwater on its SiteScope product, but don’t want to or can’t install software on your Web servers, then consider Freshwater’s SiteSeer service. This might be the case because you are hosting your Web site at a service provider who doesn’t allow these tools to be installed, or because you are running your site remotely and don’t have administrative access to the server itself. Like WebPartner, it is a Net-based service you subscribe to (but is fee-based rather than free). In addition to e-mail alerts you can receive pager notification. A free trial is available at the company’s Web site.

Site load balancing

But these tools still don’t cover the entire management picture. You also need to handle peak load traffic to your site, and be able to understand how to calculate and anticipate these peak loads properly and how they will affect your various servers and systems. Perhaps the most complete service for this situation is provided by a company called Manage.com, which will set up the appropriate servers, routers, and switches to ensure that your site will be able to satisfy the number of visitors and Web shoppers coming by, even at the busiest of times. The company has test tools to examine your Web storefront and monitor such things as response time, link integrity, and overall transaction throughput. While these services are expensive (typical prices are in the tens of thousands of dollars), if you are running an e-commerce site you can’t afford not to investigate them.

Besides Manage.com, there are a number of vendors who offer Web switchgear and ways to balance loads across multiple servers. These products are useful when you have multiple Web servers that make up a single site and want to distribute your visitors across these servers in an intelligent way, such as to send the next visitor to the least busy server or to send secure transactions (like payment and order processing) to a particular set of servers. These include companies such as Alteon Web Systems Inc., Foundry Networks Inc., iPivot (now owned by Intel Corp.), and others. To make use of these products though requires a fairly heavy investment in network infrastructure, often replacing existing routers and hubs with these products to obtain the switching features and intelligent load balancing offered. For example, to handle the increased traffic, you may have to upgrade your network to 100Mb Ethernet or install a switched Ethernet network, both of which will require expensive changes to your wiring, routers, and hubs. Again, any of these products are an investment in the tens of thousands of dollars.