Creating a Linux LVM Logical Volume on an iSCSI SAN

Dec 8, 2014 iscsi linux lun netapp san storage 2014

Recently I had an Oracle database server used by some developers that was running out of space on its data volume mounted at /u02. The volume was a simple MBR volume (think fdisk), so it couldn’t be non-destructively extended without using a third-party utility like gparted. That would have been fine, but rather than leave the volume as MBR, I decided to create a new iSCSI SAN-backed Logical Volume Manager (LVM) volume, which can be extended and resized pretty easily.

In this post, I’ll show you how to create a logical volume stored on an iSCSI SAN. Even though I did this on Red Hat Enterprise Linux 6.5 (RHEL), these steps should work on any distribution of Linux.

Creating the LUN

First, you’ll need to create the LUN on the SAN. Depending on your organization, a storage administrator may have to do this. In my case, I administer the NetApp SANs (and almost everything else) so I just created a 151 GB LUN. When using LVM, I suggest sizing the LUN a little larger than your storage requirement, about 5-10% larger.

Creating an Oracle LUN on a NetApp SAN

The next step is to rescan the SAN to pick up the new LUN.

[root@oracledb01 ~]# iscsiadm -m node -R
Rescanning session [sid: 1, target: iqn.1992-08.com.netapp:sn.112133379, portal: 192.168.96.30,3260]

If you don’t already have the SAN set as a target, you can add it with the command

iscsiadm -m node -T 192.168.96.30

, where 192.168.96.30 is the iSCSI IP of your SAN.

Next, let’s find the new block device.

[root@oracledb01 ~]# fdisk -l
...
Disk /dev/sdc: 162.1 GB, 162143404032 bytes
255 heads, 63 sectors/track, 19712 cylinders
Units = cylinders of 16065 * 512 = 8225280 bytes
Sector size (logical/physical): 512 bytes / 512 bytes
I/O size (minimum/optimal): 4096 bytes / 65536 bytes
Disk identifier: 0x00000000

I’ve cut out some of the irrelevant output, but we can see the device is /dev/sdc. Let’s run fdisk against this device.

[root@oracledb01 ~]# fdisk /dev/sdc
Device contains neither a valid DOS partition table, nor Sun, SGI or OSF disklabel
Building a new DOS disklabel with disk identifier 0xccf1891a.
Changes will remain in memory only, until you decide to write them.
After that, of course, the previous content won't be recoverable.

Warning: invalid flag 0x0000 of partition table 4 will be corrected by w(rite)

WARNING: DOS-compatible mode is deprecated. It's strongly recommended to
         switch off the mode (command 'c') and change display units to
         sectors (command 'u').

Command (m for help): p

Disk /dev/sdc: 162.1 GB, 162143404032 bytes
255 heads, 63 sectors/track, 19712 cylinders
Units = cylinders of 16065 * 512 = 8225280 bytes
Sector size (logical/physical): 512 bytes / 512 bytes
I/O size (minimum/optimal): 4096 bytes / 65536 bytes
Disk identifier: 0xccf1891a

   Device Boot      Start         End      Blocks   Id  System

The ‘p’ (print) command should yield nothing since we haven’t created a partition table yet. Let’s create a new primary partition.

Creating the Partition

Command (m for help): n
Command action
   e   extended
   p   primary partition (1-4)
p
Partition number (1-4): 1
First cylinder (1-19712, default 1):
Using default value 1
Last cylinder, +cylinders or +size{K,M,G} (1-19712, default 19712):
Using default value 19712

Now that it’s created, we need to change the type to 8e which is Linux LVM.

Command (m for help): t
Selected partition 1
Hex code (type L to list codes): L

 0  Empty           24  NEC DOS         81  Minix / old Lin bf  Solaris
 1  FAT12           39  Plan 9          82  Linux swap / So c1  DRDOS/sec (FAT-
 2  XENIX root      3c  PartitionMagic  83  Linux           c4  DRDOS/sec (FAT-
 3  XENIX usr       40  Venix 80286     84  OS/2 hidden C:  c6  DRDOS/sec (FAT-
 4  FAT16

Now, print the partition table again to verify.

Command (m for help): p

Disk /dev/sdc: 162.1 GB, 162143404032 bytes
255 heads, 63 sectors/track, 19712 cylinders
Units = cylinders of 16065 * 512 = 8225280 bytes
Sector size (logical/physical): 512 bytes / 512 bytes
I/O size (minimum/optimal): 4096 bytes / 65536 bytes
Disk identifier: 0xccf1891a

   Device Boot      Start         End      Blocks   Id  System
/dev/sdc1               1       19712   158336608+  8e  Linux LVM
Partition 1 does not start on physical sector boundary.

[amazon_link asins=’020178419X,B01C1TYTPE,1617293288′ template=’ProductCarousel’ store=’benpiperbloginline-20′ marketplace=’US’ link_id=’194723f3-f7c5-11e7-9a49-638a440dbdd7′]

Everything looks good, so we’ll write it to disk.

Command (m for help): w
The partition table has been altered!

Calling ioctl() to re-read partition table.
Syncing disks.

The partition table is created.

Creating the Physical Volume

Now it’s time to create the physical volume (PV) on /dev/sdc1. The physical volume is the link between the partition and the logical volume.

[root@oracledb01 ~]# pvcreate /dev/sdc1
  Physical volume "/dev/sdc1" successfully created

Now let’s create the volume group (VG). The volume group is a container for logical volumes. We need to specify the volume group name and the physical volume.

[root@oracledb01 ~]# vgcreate vg_oracle_u02 /dev/sdc1
  Volume group "vg_oracle_u02" successfully created

Finally, we need to create the logical volume (LV). We need to specify the volume group we want to store the new logical volume in, the size of the volume (150G), and the name of the logical volume.

[root@oracledb01 ~]# lvcreate vg_oracle_u02 -L 150G -n lv_oracle_u02

Logical volume “lv_oracle_u02” created

Now let’s verify the size and get the path.

[root@oracledb01 ~]# lvdisplay vg_oracle_u02
  --- Logical volume ---
  LV Path                /dev/vg_oracle_u02/lv_oracle_u02
  LV Name                lv_oracle_u02
  VG Name                vg_oracle_u02
  LV UUID                l54y1Z-l3w0-UDtn-STVu-m77T-sX0S-SzOBIK
  LV Write Access        read/write
  LV Creation host, time oracledb01.benpiper.com, 2014-12-08 12:42:29 -0500
  LV Status              available
  # open                 0
  LV Size                150.00 GiB
  Current LE             38400
  Segments               1
  Allocation             inherit
  Read ahead sectors     auto
  - currently set to     256
  Block device           253:3

Now let’s format the new volume as ext4.

[root@oracledb01 ~]# mkfs.ext4 /dev/vg_oracle_u02/lv_oracle_u02
mke2fs 1.41.12 (17-May-2010)
Discarding device blocks: done
Filesystem label=
OS type: Linux
Block size=4096 (log=2)
Fragment size=4096 (log=2)
Stride=1 blocks, Stripe width=16 blocks
9830400 inodes, 39321600 blocks
1966080 blocks (5.00%) reserved for the super user
First data block=0
Maximum filesystem blocks=4294967296
1200 block groups
32768 blocks per group, 32768 fragments per group
8192 inodes per group
Superblock backups stored on blocks:
        32768, 98304, 163840, 229376, 294912, 819200, 884736, 1605632, 2654208,
        4096000, 7962624, 11239424, 20480000, 23887872

Writing inode tables: done
Creating journal (32768 blocks): done
Writing superblocks and filesystem accounting information: done

This filesystem will be automatically checked every 38 mounts or
180 days, whichever comes first.  Use tune2fs -c or -i to override.

Now let’s create a mount point and mount the new volume.

Mounting the Volume

[root@oracledb01 ~]# mkdir /u02new
[root@oracledb01 ~]# mount /dev/vg_oracle_u02/lv_oracle_u02 /u02new/

Now let’s verify the volume is mounted and has the expected amount of free space.

[root@oracledb01 ~]# df /u02new/ -h
Filesystem            Size  Used Avail Use% Mounted on
/dev/mapper/vg_oracle_u02-lv_oracle_u02
                      148G   60M  140G   1% /u02new

Looks good. And that’s it!