As the bandwidth demands on campus networks continue to grow and the cost of campus network devices continues to drop, many companies are looking towards switches and virtual LANs (VLANs) to upgrade their network infrastructure. Once the decision is made to move towards a switched network, network managers need not only to understand how VLANs work, but also must have a grasp of issues related to sound VLAN network design.
Through the course of this article, I hope to introduce you to the basics of both a VLAN and a VLAN-based campus switched network. Then we will look at some design issues related to VLANs such as redundancy and load balancing.
At its most basic level, a VLAN is analogous to a stack of repeater hubs. It is a broadcast domain. It is a subnet. It is a logical grouping of switch ports into a single entity. Is it truly possible for a VLAN to be all this? The answer is yes.
VLANs can be made up of any number of ports in the campus switched network. It can be all ports in the network, it can be only one port in the network, and it can be any number of ports in between. This grouping of ports is accomplished by the switches communicating information concerning each VLAN between them. In the Cisco world, the VLAN Trunking Protocol (VTP) accomplishes this communication. VTP allows each switch in the network to tell every other switch which VLANs have been programmed into its own database. By this method, each and every switch has a complete listing of the available VLANs that can be assigned to individual switch ports.
This allows network managers to move their user servers from these remote locations into a central server room, which can be secured, well ventilated, and central to the core of the network. In addition to easing communication between buildings, VLANs also ease communications within the same building. Having multiple user groups on multiple floors no longer requires network managers to maintain multiple repeater hub stacks and multiple router ports per floor regardless of what software is being run such as graphic design platforms.
User data from two PCs on the same VLAN who are connected to two different switches in the network is carried on one physical connection between those two switches. Both the transmitting and receiving switches are able to determine which VLAN any one packet belongs to by using a trunking method called Inter-Switch Link (ISL). ISL is a proprietary Cisco technology where two Cisco switch devices (or a switch and a Cisco router as we will see later) communicate VLAN information to each other.
ISL appends a header onto every frame it sends to the neighboring switch. This header tells the neighbor switch which VLAN this frame belongs to. ISL headers are only attached to data frames between devices. The outgoing trunk port on the source switch attached the header and the incoming trunk port of the destination switch strips the header back off.
With all the data flowing through the switches in the same VLAN, how do devices on different VLAN communicate? LAN to LAN communication has traditionally been the role of the router and this same device continues to play the same role in a campus switched network. Remember that each VLAN is a subnet and inter-subnet communication uses Layer 3 of the OSI model. Now, how are routers used in a switched network?
Similar to repeater hubs, each VLAN requires a connection to a router. On a basic level, each VLAN needs one dedicated connection to a router port. This can quickly consume router ports and might cause some networks to purchase more powerful routers and more router ports than they really need. Cisco has devised two alternative options to this problem. The first option is to connect one of the router’s Fast Ethernet ports to one port on the Catalyst switch.
These two devices can then communicate with each other using the same ISL encapsulation that the switches use to communicate. From the routerÕs perspective, the Fast Ethernet port is configured to use sub-interfaces, one for each subnet. The sub-interfaces function exactly as do “real” Ethernet interfaces.
Traffic between VLANs now travels from the switch to the router and back again on the same physical wire. Logically, the data goes to the router with an ISL header of A and is received on one sub-interface. The router then routes the packet to a different sub-interface and sends the packet back to the switch with an ISL header of B (Figure 1).
The second option is to install a Cisco Route Switch Module (RSM) into one of your Catalyst switches. The RSM installs directly into the Catalyst chassis and attaches to the backplane. It has one virtual interface configured for each VLAN present in the network. Functionally, the RSM operates exactly like the Fast Ethernet ISL connection we just described. The main difference between the two is that the RSM can route between VLANs at Catalyst backplane speeds (1.2 Gbps) as opposed to Fast Ethernet Full Duplex speeds.
Cisco has recently announced a more efficient routing mechanism for use in a switched campus network. The Catalyst Supervisor III switching engine can now be supplemented by a daughter board upgrade called the NetFlow Feature Card (NFFC). The inclusion of this card allows the Catalyst switch and either of the router options discussed above to share layer 3 IP addressing information.
Redundancy in a campus switched network with Cisco networking devices can be provided in many different ways. One option is to place two Supervisor III switching engines into the same Catalyst 5500 series chassis. One of these modules will function as the master and the other as the slave. Should the master fail for any reason, the slave takes over the operation of the switch. A second redundancy option available is to have two different routers running in your network. These two routers can appear as one router to user PCs by utilization of the Hot Standby Router Protocol (HSRP).
HSRP routers (called primary and standby) communicate between themselves and agree to use both a virtual IP address and MAC address so that user PCs can be statically configured for one default gateway. In this manner, data traffic can be guaranteed to flow through the network because the failure of the primary router causes the standby router to start processing data traffic destined for the virtual address.
One possible resiliency option for switches in a campus network is to have multiple network links between each switch (Figure 2).
From a physical standpoint, these network links should take different paths throughout the campus. On a logical layer, these multiple links can be used to load balance network traffic between the switches by adjusting different settings of the Spanning Tree Protocol (STP).
STP is the method by which switches in a campus network guarantee a loop-free topology. STP was designed to overcome one of the most basic traits of switches. This trait causes all switches to forward each and every broadcast out of all of its ports. STP makes sure that only one broadcast packet is seen on each network link by electing a root of the network. Each switch then traces a least-cost path back to the root. Under normal circumstances, this configuration forces all user traffic along the same set of wires wasting possible network bandwidth.
CiscoÕs STP and VLAN implementation provides for one instance of STP for each VLAN in the network. This feature allows a network manager to assign different roots for each VLAN. Additionally, managers may also decide which physical wires carry traffic for which VLANs. These options allow more bandwidth in the network to be utilized.
By default, all Catalyst switches have every port placed into VLAN 1; that VLAN is the only one running on the switch. This works well for a plug and play situation, but for an operational network (even the smallest) there should be a minimum of 2 VLANs. The default VLAN 1 should be used as a management VLAN. This means that the IP address of each of the switches in your network should be placed into this VLAN. This allows network staff to telnet to the switches without being hampered by or interrupting user data.
In addition, this also allows Simple Network Management Protocol (SNMP) traffic to flow throughout the network with much more ease. You should then create additional VLANs and place user PCs and/or servers into them. This begs the question of how many devices should be in each VLAN.
The answer to the question of VLAN size is best measured by the traffic patterns of each particular network. Factors such as a need to see broadcast traffic, groupings of users, and logical arrangement of network segments all play a part. In general practice, about 150 devices in a VLAN is a good upper limit. Network designers should not over-compensate in the opposite direction and use a large number of smaller VLANs either. Remember that each VLAN keeps a separate version of the spanning tree.
The more VLANs you place on a switch, the higher the load you are placing on the CPU of each and every switch in the network. In a large production environment of 3000-5000 nodes, around 30-35 VLANs is a good upper limit.
When it comes to a physical layout of your switched network, one of the most successful implementations is a hub and spoke design. This allows network staff to place a large switch like a Catalyst 5500 at the center of the network to function as the core switch and to be the STP root for all of the VLANs in the network. This works well due to the fact that each remote switch is now only one data hop away from the root of the spanning tree and two hops away from any other switch in the network.
To provide redundancy options for the network core switch, a second Catalyst 5500 series switch can be placed at the core. For added redundancy, the second switch should at a minimum be connected to a separate power circuit. The second switch can also be placed in a different room on the floor or on an entirely different floor altogether. Each closet switch would then have two data connections outbound from it to the core, one for each core switch (Figure 3).
Communications between the two core switches should be accomplished via ISL trunking and Fast EtherChannel (FEC).
FEC is a method where two different physical links between two network devices are combined logically together to ÒdoubleÓ the bandwidth. This comes into play with two core switches in the following manner. Under normal circumstances, one of the two wires would not be used for data traffic due to STP preventing bridging loops by blocking one of the ports. When FEC molds the two lines onto one, STP sees this wire as one unit and all data traffic now flows over both physical lines between the switches.
For those networks that require redundancy options at the user PC level, network designers can place two PCs on each desk and place each of these PCs in a different VLAN. One such environment might be a securities trading company whose brokers need mission critical information every moment of the day. A database front-end can be placed on servers in each and every VLAN. Each of the servers would be fed its information from a common back-end server and/or mainframe. One benefit of this configuration is that the client/server information is a UDP broadcast every 2-5 seconds, and placing servers in every VLAN reduces the broadcast traffic across the network as a whole. Now, these multiple PCs can ensure the end-userÕs access to mission critical information in the event of a failure in any one VLAN.
VLANs are a technology that is rapidly changing the way in which networks are built. It is important that designers know the basic capabilities of VLANs and factors that play into their operation. Building on these blocks will allow network staff to configure a highly redundant, stable, and powerful network.