Storage Area Network

Challenges with Directly Attached Storage:

1.  Storage remains isolated and underutilized.
2.  Complexity in sharing storage resources across multiple servers.
3.  High cost of managing information.
4.  Challenges in scalability.

An effective information management system must provide:

1. Timely information to business users
2. Flexible and resilient storage infrastructure.

A storage area network(SAN) provides such a solution.

A storage area network is a high-speed, dedicated network designed to deliver block-level storage to computers that are not directly connected to the storage devices or drive arrays. The storage in a SAN is not owned by any server unlike DAS(Directly Attached Storage) but is accessible by all of the servers on the network.

Advantages of SAN:

Enables sharing of storage resources across multiple servers.
Centralizes storage and management
Meets increasing storage demands efficiently with better economics of scale.

SAN Classification:

• Fibre Channel (FC) SAN:                            uses Fiber Channel protocol for communitcation.
• IP SAN:                                                        uses IP-based protocolss for communication
• Fibre Channel over Ethernet (FCoE) SAN: uses FCoE protocol for communication.

Understanding Fibre Channel:

High-speed network technology: Supports upto 16 Gbps

Highly Scalable : accomodates approximately 15 million devices.

Components of FC SAN:

Node (server and storage) ports: Provide physical interface for communicating with other nodes.

Exist on 

    - HBA in server

    - Front-end adapters in storage

Each port has a transmit(Tx) link and a receive (Rx) link

Cables:  

SAN implementation uses

    - Optical fiber cables for long distances

    - Copper cables for short distance

Two types of optical cables: 

Single-mode: Carries single beam of light and carries signal upto 10 km

Multimode   : Can carry multiple beams of light simultaneously. Used for short distance

Connectors: 

Attached at the end of a cable

Enable swift connection and disconnection of the cable to and from a port

Commonly used connectors for fiber optic cables are:

      Standard Connector(SC): Duplex connectors

      Lucent Connector(LC) : Duplex connectors

      Straight Tip(ST) : Patch panel connectors and Simplex connectors.

Interconnecting Devices:

Commenly used interconnecting devices in FC SAN are:

     - Hubs, switches and directors

Hubs provide limited connectivity and scalability

Switches and directors are intelligent devices

    - Switches are available with fixed port count or modular design   

    - Directors are always modular, and its port count can be increased by inserting additional 'line             cards' or 'blades'.

    - High-end switches and directors contain redundant components.

    - Both switches and directors have management port to connect to SAN management servers.

SAN Management Software:

    - A suite of tools used in a SAN to manage interfaces between host and storage arrays

    - Provides intergrated management of SAN environment

    - Enables web-based management using GUI or CLI

FC Interconnectivity Options:

Point-to-Point Connectivity: 

• Simplest FC configuration which enables direct connection between nodes.
• Offers limited connectivity and scalability
• Used in DAS environment

FC-AL Connectivity:

• Provides shared loop to attached nodes: Nodes must arbitrate to gain control
• Implemented using ring or star topology. May also use hub which uses star topology.
• Limitations of FC-AL :

                - Only one device can perform I/O operation at a time

                - Uses 8 bit of the 24 bit fiber channel addressing.1 address is reserved to connect to FC                          switch port. Supports upto 126 nodes.

                - Addition or removal of a node causes momentary pause in loop traffic

FC- SW Connectivity :

• Creates a logical space(called fabric) in which all nodes communicate using switches. Interswitch links(ISL) enable switches to be connected.
• Provides dedicated path between nodes.
• Addition/removal of node does not affect traffic of other nodes.
• Each port has unique 24 bit FC address.

Port Types in Switch Fabric:

Port provides physical interface to a device to connect to other devices. The types are:

N_port: or Node port is typically a host port of storage array switch.

E_port: or Extension port which is connected to E-port of other switch

F_port : or Fabric port is a port in switch which connects to N_port

G_port: or Generic port can work as F_port or  E_port which is automatically done.

Fibre Channel Protocol (FCP) Overview:

• Traditional technologies such as SCSI have limited scalability and distance
• Network technologies provide greater scalability and distance but have high protocol overhead.
• FCP provides benefits of both channel and network technologies
•      - High performance with low protocol overheads
•      - High scalability with long distance capability
• Implements SCSI over FC network
• Storage devices attached to SAN, appear as local storage devices to host operating system

Addressing in switched Fabric:

The server or disk array which has a HBA reports itself to the network using Fabric Login(FLogi). It advertises its NWWN(Node World Wide Name). The FC switch replies with the FC ID for that device. This functionality is similar to that of a DHCP.

A FC switch has a block of addresses assigned to it represented by its Domain ID.

Domain ID is a unique number provided to each switch in the fabric. Domain IDs can be statistically or dynamically configured. Since permission is required to assign domain ID, it never overlaps. One switch is elected as the principal switch. This is elected based on priority value and system WWN. The lowest one wins. No backup principal switch is elected unlike DR/BDR selection in normal switches. If the principal switch dies, new now is elected. The failover is fast.

    - 239 addresses are available for domain ID.

Maximum possible number of node ports in a switched fabric:

    - 239 domains * 256 areas * 256 ports = 15,663,104

In case of multiple switches, FSPF(Fabric Shortest Path First) is used for routing which uses Fabric IDs. Fiber channel routing table is checked for routing.

Address Resolution using Fiber Channel Name Server(FCNS):

FCNS has a list of PWWN and FC ID. This server is run by principal switch. FCNS database is distributed across all switches so there is no need of backup. As soon as the device gets reply from switch with the FC ID, the host will send PLogi message with PWWN and FC ID thus registering itself with principal switch. For address resolution, host will send query to FCNS with PWWN and FCNS replise with FC ID. Thus routing is based on FC ID which is a logical address. Thus fiber channel is a layer-3 protocol.