SAN and iSCSI Design: Targets, Initiators, Multipath and Failure Domains

1. The basic iSCSI model

The storage system exposes one or more targets. A host runs an initiator and connects to those targets across IP. Logical units are then presented to the operating system. Treat the storage network as infrastructure, not as an ordinary user LAN with a few extra packets.

2. Separate availability from bandwidth

Two NICs can provide additional paths, but only if the complete design includes independent switch paths and correct multipath configuration. Two cables into one switch still share that switch as a failure point.

3. IP and VLAN design

Use a documented addressing plan. Dedicated VLANs or physically separate networks can simplify control and fault isolation. Jumbo frames are not mandatory for iSCSI and should only be used when the complete path is configured and validated consistently.

4. Multipath

Multipath software can maintain multiple paths to the same storage and select or fail over between them. Vendor-supported policy matters. Do not configure multiple independent filesystem mounts to the same block device as a substitute for multipath.

5. Authentication and access control

Restrict which initiators can access which targets. CHAP may be used where supported, but network segmentation, host permissions and storage-side mapping are also important. Never expose storage targets broadly without an access design.

6. Performance troubleshooting

Measure host latency, queueing, packet loss, NIC errors, switch counters and array latency. A slow VM datastore can be caused by storage media, congestion, path failover, host queueing or backup activity.

7. Failure testing

For a redundant design, deliberately test one path at a time during commissioning. Disconnect a storage path or disable a switch port according to an approved test plan and verify that I/O continues as designed.

8. Commissioning checklist

• Record the exact model and firmware/software level.
• Verify physical interfaces and supported media.
• Confirm power, cooling and environmental requirements.
• Test the intended workload, not only basic connectivity.
• Record baseline measurements and configuration.
• Label and document the installation for future fault finding.

Field application and acceptance

This section turns the technical concepts above into a repeatable procurement, installation and handover process.

Field design workflow

Build the design from measured workload data where possible. Record current used capacity, growth rate, read/write pattern, peak latency sensitivity, backup window and recovery objective. Identify every host that will consume the storage and the protocol it expects. Then draw the complete path from application to media, including controller, network and protection layers.

Only after that should you compare products. This prevents a common procurement mistake where a system is selected because it has enough terabytes while the interface, controller, network or availability design is wrong for the workload.

Questions to answer before ordering

• What exact workload will live here?
• How much usable capacity is required today and after planned growth?
• What protection level and recovery time are required?
• Which hosts, operating systems and hypervisors must be supported?
• What network or direct-connect interfaces are required?
• What drive types and capacities are supported by the selected platform?
• How will backups be created, tested and restored?
• What is the expansion or migration plan when capacity is exhausted?

Fault-finding method

Do not begin by replacing disks. Separate the problem into host, path, controller, array and media layers. Check operating-system latency and queueing, adapter status, switch counters where applicable, storage-controller events, array health and drive alerts. Compare the fault period with backup, snapshot, replication or rebuild activity.

A useful baseline taken during commissioning makes later diagnosis much faster because you know what normal latency, throughput and utilisation looked like before the fault.

Operational documentation

Record storage model, serials, firmware, drive map, RAID or protection layout, pool/volume names, network addresses, initiators/targets, multipath policy, backup jobs, snapshot policy and alert destinations. For shared storage, document which hosts are authorised to see each volume.

What correct looks like

The installed system presents the expected usable capacity, all drives and paths are healthy, protection is active, alerts reach the correct people, backups complete inside the required window and a test restore has been performed. Performance measurements are saved as a baseline and there is a documented response for drive failure, path failure and capacity growth.