Creating a Linux LVM Logical Volume on an iSCSI SAN

Recently I had an Oracle database server used by some developers that was running out of space on its data volume mounted at /u02. The volume was a simple MBR volume (think fdisk), so it couldn’t be non-destructively extended without using a third-party utility like gparted. That would have been fine, but rather than leave the volume as MBR, I decided to create a new iSCSI SAN-backed Logical Volume Manager (LVM) volume, which can be extended and resized pretty easily.

In this post, I’ll show you how to create a logical volume stored on an iSCSI SAN. Even though I did this on Red Hat Enterprise Linux 6.5 (RHEL), these steps should work on any distribution of Linux. Continue reading

How to Make NetApp Use the Correct Interface for iSCSI

If you’re familiar with networking you know that when a device is directly connected to two separate IP networks, traffic destined for one of those networks should egress on the interface that is directly connected to that network. For example, if your storage appliance is directly connected to the 172.16.1.0/24 network, and you want to send a packet to a device with the IP of 172.16.1.55, traffic should egress on the interface connected to that network. Unfortunately, in the case of some NetApp filers, this does not always happen.

I ran into a peculiar issue when trying to force NetApp’s Snapmirror to replicate across a specific interface, only to be met with an ugly “Snapmirror error: cannot connect to source filer (Error: 13102)”. I confirmed with NetApp support that the Snapmirror configuration was correct for what I was trying to accomplish.

To troubleshoot, I started a packet trace on the destination filer using the command:

pktt start all -d /etc

I then kicked off the snapmirror initialization, waited for it to fail, then stopped the packet trace with

pktt stop all

Since I directed the trace files to be placed in /etc on the filer I just browsed to the hidden etc$ CIFS share on the filer and opened the traces in Wireshark. What I found was that the traffic that should have been egressing on the iSCSI VIF was actually going out on the LAN VIF. Not only that, the filer was using its iSCSI address on the LAN VIF! I’m always hesitant to label every quirk a “bug,” but this is definitely not correct behavior.

The remedy was as simple as adding a route statement similar to this:

route add inet 172.16.1.0/24 172.16.2.1 1

where 172.16.1.0/24 is the iSCSI network I want to traverse to reach the Snapmirror partner, and 172.16.2.1 is the gateway on my locally connected iSCSI network. The 1 specifies the cost metric for the route, which will always be 1 unless you need to add additional gateways.

To make the change permanent, simply add the route statement to the /etc/rd file on the filer.

Special thanks to NetApp’s Scott Owens for pointing me in the right direction on this.