For decades, the backbone of SCADA communications networks has been the lowly telco leased line. Provided in two-wire or four-wire format with either point-to-point or multi-drop configurations, these were the vital lines of communication between remote site control points and central management control centers. Supplementing these where economy, reliability, or availability issues came into play were private point-to-point or multi-point VHF (and then UHF) radio and point-to-point microwave links. This equipment was (eventually) reliable, economical, and almost always under the total control of the operator.

In todays world however, a SCADA network may be built from many possible communications solutions which include all those old standards as well as Ethernet WANs, Fiber Optics, several license-free ISM microwave bands, ATM, DDS, Frame Relay, and T-1 connections. A properly designed system should make use of the most appropriate mix of these technologies to meet design goals.

First, a list of the technologies and then we'll describe them in more detail.

• Analog Point-to-point and multi-point modem lines
• DDS
• Frame Relay point-to-point and mesh networks
• ATM
• T-1 and higher

Fiber:

• SONET or Fiber Ring
• Fiber Ethernet

• Wireless Common Carrier networks
• VHF and UHF point-to-point and multi-point
• ISM Band License-Free ( 900 Mhz, 2.4 Ghz, 5.8 Ghz, 24 Ghz)
• Point-to-point Microwave (23 Ghz)

Analog Modems: Long the mainstay of WAN communications, the leased analog phone line is rapidly disappearing. Telephone companies are de-tarriffing them as fast as they can since they often cost the telco more in facilities and maintenance than they produce in revenue. We hear from many customers that they simply can't get analog lines where they need them something that was unheard of just a few years ago. Other than that "political" problem; analog lines are limited in bandwidth, with most leased lines running at relatively slow rates of 1200, 4800, or 9600 bps either asynchronous or synchronous. Limitations on the technology place leased line maximum speeds at about 9.6 Kbps for multi-drop lines and around 14.4 or 19.2 Kbps for point-to-point lines. (The 56Kbps dial-up modems are not for real, they don't actually have that much bandwidth, and they only provide increased bandwidth when talking to a special modem located at the central office closest to the modem. Since the Internet revolution, most analog modems are consumer grade, but a few companies still manufacture an industrial grade modem. Very few of us still manufacture multi-drop modems. If available, this is the "bottom end" of the communications spectrum.

DDS: Dataphone Digital Service is the 56 or 64 Kbps digital leased line. These are commonly available, and provide a medium speed synchronous connection. By using the proper DSU equipment at the customer premises, the line can be used for lower speed asynchronous connections. These circuits are mostly trouble-free, and are the garden-variety connection for the computer industry. They sometimes cost more than SCADA users want to pay for the slow connections that they often need. They are available in point-to-point or multi-point; but few customers opt for multi-point DDS. Often a good fit for installations that require medium bandwidth over tens or hundreds of miles.

Frame Relay: This is a packet switched mesh network that is popular for WAN connections between Ethernet LANs. It's becoming popular with SCADA operators because of it's low cost and some inherent features such as built-in redundancy and scaleable bandwidth. It's also the economical fit for longer distance networks and networks that span telephone company LATAs. A tutorial on frame relay is available for download at HTTP://www.dcbnet.com/notes/0004framerelay.html .

ATM: Asynchronous Transfer Mode is a high-speed cell-switching technology that operates from 1.5 Mbps up through the gigabit range. It is normally too expensive for SCADA applications due to this high bandwidth and access devices that are too expensive for RTU applications. ATM networks usually provide much more bandwidth than required for SCADA and at a much higher cost. If a ultra-high bandwidth is needed say 100 Mbps, this might be the way to go. It would require IP type protocols and special access devices.

T-1 and Higher: T1, T3, OC-3 and such high speed digital connections are becoming more common. Prices are dropping for these high speed connections and they are being used for computer networks that relied upon DDS circuits just a year or two ago. Speeds begin at 1.5 Mbps and go up, so there is lots of bandwidth available often much more than needed by simple SCADA systems. But, if an Ethernet connection is really needed, T-1 is often the bottom end speed for office LAN connections. An excellent T-1 tutorial is available for download at HTTP://www.dcbnet.com/notes/9611t1.html .

Fiber & SONET: The fiber technologies are also a high bandwidth option. Although the infrastructure cost is low (fiber is actually inexpensive to install), there is high cost equipment needed to terminate the fiber which raises network cost considerably. If the telco will provide an interface device at reasonable cost, and your RTUs speak IP over Ethernet, this might be worth considering. One exception to this high-cost rule is the Fiber Modem. Since it's economical to pull extra fiber when installing private fiber for other uses, DCB manufactures a low-cost, low speed fiber modem that transports RS-232 data over fiber.

Fiber Ethernet: In some cases, Ethernet can be transported by lower-cost fiber without using the SONET technology. This is the common factory-floor and Fiber LAN application. If the WAN is small enough, this is a good solution to provide Ethernet speeds over relatively short distances. Again, the RTUs need to use IP over Ethernet.

VHF and UHF Radio: This is a privately owned radio system. Commonly used where terrain and lack of phone lines are a problem, and capable of only 19.2 Kbps or less throughput. These radio frequencies require licensing, and are no longer available in many locations due to radio frequency congestion. If you have one now, you might consider expanding it if the bandwidth is adequate. If you don't already have a VHF/UHF radio network, ISM band equipment is often a better fit with higher bandwidth and lower cost.

Wireless common-carrier networks: CDPD and public wireless Internet have not covered the country yet and are often missing in our most critical geographic areas. They tend to cover the cities and Interstates while ignoring the back roads and rural landscape. These also come with a per-packet cost or cost per Kb transferred. Usually available only in IP protocols, but, they are easy to implement where they fit. Unfortunately, bandwidth is severely limited often to 9600 bps or even slower.

ISM Band Radio: ISM band equipment comes in many flavors ranging from point-to-point and multi-point radios running at 9.6 Kbps all the way up to point-to-point radios at 11 Mbps. Range varies with frequency We have installed 900 Mhz units with 50 mile ranges, but some 2.4 Ghz or 5.8 Ghz units max out at just a few miles. Some of these systems are relatively simple and forgiving, others require extensive engineering design expertise to build a reliable system. This equipment does not require a license (other than manufacturer's type acceptance), and interference is "at the risk" of the system owner. For many common SCADA requirements, this is a near ideal solution for networks that span distances of up to tens of miles. But, the at-risk nature of license free equipment prevents many from using it. Several good references may be downloaded from HTTP://www.dcbnet.com/notes/9805wire.html

Point-to-Point Microwave: This is just that licensed microwave radio equipment operating on a frequency assigned to a single user. Frequencies are commonly 23 Ghz and higher (although lower frequencies are still in use). Bandwidth is typically T1 speeds of 1.5 Mbps or higher, with 10 Mbps being most common. This is the most secure and reliable of the wireless methods; but it also requires link engineering and time to obtain licenses.

The Organization Office WAN: Not a technology, but worthy of mention here because it's becoming more common, and there is lots of financial pressure to add SCADA to it. Once an organization installs an ethernet WAN to network all the offices together, management seems to think that adding SCADA to that network is an economical way to go. Unfortunately, there are some downside effects that must be considered The corporate network is often not as robust and reliable as the most elemental SCADA network. It's operated with different criteria by a different department. For example, an unannounced midnight network shutdown for maintenance may be fine for the office workers; but might create alarms and possibly backed-up sewage for the wastewater department. If you consider this method tread carefully.

Those are the technologies, how does one decide which to use? It takes careful analysis and a bit of detective work.

1. Always analyze for the life-cycle cost. Determine an effective lifetime for the network and analyze costs, risks, and benefits for the entire life cycle. Normally, installed cost is only a small part of the equation.
2. Determine actual bandwidth requirements. If a SCADA network is used for alarm reporting and a few control points, a simple 9.6 Kbps network may be all that is needed. If complex reporting, real-time video, multiple operation centers, and life-safety is involved, more bandwidth and redundancy is required. But, don't forget that bandwidth requirements always seem to grow with time. A thorough bandwidth needs analysis will normally eliminate many of the options
3. Look at your in-house expertise. A user-owned microwave or ISM radio network may be impressive. But, do you have the in-house expertise to install and maintain it? Are there vendors that you can depend upon for those functions? Do your people understand Frame Relay or ATM? Perhaps some basic education is needed before the analysis begins.
4. What is the track record for your vendors? Is the telephone company notorious for outage times and MTTR? Do they jump through loops to make you happy?
5. What are the physical and geographic criteria? Mountains may prevent ISM equipment from working, or they may provide the high location from which all remotes may be reached.
6. What does the system need to grow into next year? How about the next 5 years?

What mixture of communication technologies provides the optimum solution to these questions?

A successful SCADA WAN network is usually a compromise between reliability, cost, bandwidth, and flexibility. Today, more than ever, SCADA network operators are required to economize while still providing the necessary control and data acquisition functions. Most SCADA networks contain a technology mixture that grew in complexity over the years. Make sure that your analysis allows for even faster growth in the future.

Examples:

Traffic control in an east coast town was coordinated with multi-drop modems. When the city installed a fiber optic network to support other utility operations, the control system was re-evaluated and some modems were replaced with DCB Etherpath ethernet to RS-232 gateways. The resulting network used some point-to-point modems, some multi-drop modems, and some high speed nodes with fiber Ethernet interfaces.

When a mountain state utility needed to control a wastewater pump station that was beyond local telephone lines, they opted to install a 900 Mhz ISM band radio modem to connect the RTU to their existing system. The rest of their system uses point-to-point and multi-drop modems.

A regional electric utility selected frame relay instead of analog modem lines for their new installations. Using the frame relay network, they were able to provide a back-up operations center full control with a small incremental cost.

Reference:

The following white papers may be freely downloaded from the education section at DCB's web site: http://www.dcbnet.com .

An overview of Frame Relay communications and how it is useful for SCADA communications. This white paper details the way Frame Relay can be used with asynchronous polling SCADA RTUs with no additional protocol needed on the RTU or host computer.

Customers often ask "What is the reliability of Frame Relay versus analog phone lines?". In our experience, the Frame Relay service has been more reliable. This discussion presents some evidence and technical information that supports this improved reliability.

Some rules of thumb that can help you make a good wireless decision. Basic, required reading for everyone considering that first wireless link.

A Paper describing Frame Relay originally delivered to the Third Annual Traffic Communications Seminar, Tahoe, October 1995 and updated in March, 1997.

Globally Serial: Extending the Reach of your Serial Devices Around the World"

An article that recently apeared in information technology trade journals. This article discusses the use of DCB's EtherPath external device servers and how they are used to connect RS-232 serial devices using a wide area ethernet network.

A complete explanation of T1 in all its gory details from basics to signaling. Written for our distributors and rather deep.

Data Comm for Business Inc. 2949 County Road 1000 E Dewey, Il 61840	Voice: 217-897-6600 Toll Free: 800-4-DCB-NET Toll Free: 800-432-2638	Email: Contact Page Web: www.dcbnet.com Fax: 217-897-8023
All DCB web pages copyright ©1995- Data Comm for Business, All rights reserved. EtherPath®, EtherSeries®, EtherPoll®, EtherBridge® and EtherModem® are Registered Trademarks of Data Comm for Business, Inc.