How to Make NetApp Use the Correct Interface for iSCSI

Jan 25, 2012 iscsi netapp networking snapmirror storage 2012
Share on:

If you’re familiar with networking you know that when a device is directly connected to two separate IP networks, traffic destined for one of those networks should egress on the interface that is directly connected to that network. For example, if your storage appliance is directly connected to the 172.16.1.0/24 network, and you want to send a packet to a device with the IP of 172.16.1.55, traffic should egress on the interface connected to that network. Unfortunately, in the case of some NetApp filers, this does not always happen.

I ran into a peculiar issue when trying to force NetApp’s Snapmirror to replicate across a specific interface, only to be met with an ugly “Snapmirror error: cannot connect to source filer (Error: 13102)”. I confirmed with NetApp support that the Snapmirror configuration was correct for what I was trying to accomplish.

To troubleshoot, I started a packet trace on the destination filer using the command:

1pktt start all -d /etc

I then kicked off the snapmirror initialization, waited for it to fail, then stopped the packet trace with

1pktt stop all

Since I directed the trace files to be placed in /etc on the filer I just browsed to the hidden etc$ CIFS share on the filer and opened the traces in Wireshark. What I found was that the traffic that should have been egressing on the iSCSI VIF was actually going out on the LAN VIF. Not only that, the filer was using its iSCSI address on the LAN VIF! I’m always hesitant to label every quirk a “bug,” but this is definitely not correct behavior.

The remedy was as simple as adding a route statement similar to this:

1route add inet 172.16.1.0/24 172.16.2.1 1

where 172.16.1.0/24 is the iSCSI network I want to traverse to reach the Snapmirror partner, and 172.16.2.1 is the gateway on my locally connected iSCSI network. The 1 specifies the cost metric for the route, which will always be 1 unless you need to add additional gateways.

To make the change permanent, simply add the route statement to the /etc/rd file on the filer.

Special thanks to NetApp’s Scott Owens for pointing me in the right direction on this.