systemadmin.es > Storage > SAN > Instalación de iSCSI con una Dell EquaLogic (CentOS 5)

Instalación de iSCSI con una Dell EquaLogic (CentOS 5)

Vamos a ver como configurar el iniciador iSCSI con multipath en RHEL/CentOS 5 contra una cabina Dell EquaLogic. También disponible para CentOS 6.

Primero de todo deberemos cambiar la MTU y habilitar el flow control en la interfaz. Supondremos que las interfaces que vamos a usar para iSCSI son eth2 y eth3. Para ello deberemos modificar los ficheros /etc/sysconfig/network-scripts/ifcfg-eth2 e ifcfg-eth3 añadiendo:

(...)
ETHTOOL_OPTS="autoneg on rx on tx on"
MTU=9000

Para cambiar la MTU y habilitar flow control a mano lo haremos con ip y ethtool:

ip link set eth2 mtu 9000
ip link set eth3 mtu 9000
ethtool -A eth2 autoneg on rx on tx on
ethtool -A eth3 autoneg on rx on tx on

A continuación con ethtool podemos verificar que hemos habilitado flow control:

# ethtool -a eth2
Pause parameters for eth2:
Autonegotiate:	on
RX:		on
TX:		on

Podemos proceder comprobando la conectividad iSCSI haciendo un ping a la cabina desde cada interfaz:

[root@sunfire log]# ping -I eth0 192.168.1.10
PING 192.168.1.10 (192.168.1.10) from 192.168.1.15 eth0: 56(84) bytes of data.
64 bytes from 192.168.1.10: icmp_seq=1 ttl=255 time=1.62 ms

--- 192.168.1.10 ping statistics ---
1 packets transmitted, 1 received, 0% packet loss, time 0ms
rtt min/avg/max/mdev = 1.629/1.629/1.629/0.000 ms
# ping -I eth1 192.168.1.10
PING 192.168.1.10 (192.168.1.10) from 192.168.1.16 eth1: 56(84) bytes of data.
64 bytes from 192.168.1.10: icmp_seq=1 ttl=255 time=0.887 ms

--- 192.168.1.10 ping statistics ---
1 packets transmitted, 1 received, 0% packet loss, time 0ms
rtt min/avg/max/mdev = 0.887/0.887/0.887/0.000 ms

Procedemos instalando el paquete iscsi-initiator-utils:

yum install iscsi-initiator-utils

Y habilitando los daemons al inicio del equipo:

chkconfig --add iscsi
chkconfig iscsi on
chkconfig --add multipathd
chkconfig multipathd on

Podemos verificar con chkconfig –list:

# chkconfig --list | grep -E iscsi\|multi
iscsi          	0:off	1:off	2:on	3:on	4:on	5:on	6:off
iscsid         	0:off	1:off	2:off	3:on	4:on	5:on	6:off
multipathd     	0:off	1:off	2:on	3:on	4:on	5:on	6:off

A continuación podemos probar usando un único camino a la cabina, haciendo primero el discovery:

# iscsiadm -m discovery -t st -p 192.168.1.10
192.168.1.10:3260,1 iqn.2001-05.com.equallogic:0-dead00-7cb4e0808-c4b000805b04f34d-fuckyeah

Y posteriormente haciendo el login:

# iscsiadm -m node -T iqn.2001-05.com.equallogic:0-dead00-7cb4e0808-c4b000805b04f34d-fuckyeah -p 192.168.1.10 --login
Logging in to [iface: default, target: iqn.2001-05.com.equallogic:0-dead00-7cb4e0808-c4b000805b04f34d-fuckyeah, portal: 192.168.1.10,3260]
Login to [iface: default, target: iqn.2001-05.com.equallogic:0-dead00-7cb4e0808-c4b000805b04f34d-fuckyeah, portal: 192.168.1.10,3260] successful.

Mediante -m session podemos ver las sesiones activas:

# iscsiadm -m session 
tcp: [1] 192.168.1.10:3260,1 iqn.2001-05.com.equallogic:0-dead00-7cb4e0808-c4b000805b04f34d-fuckyeah

Para seguir deberemos hacer logout de la sesión de prueba que hemos establecido:

# iscsiadm -m node -u
Logging out of session [sid: 1, target: iqn.2001-05.com.equallogic:0-dead00-7cb4e0808-c4b000805b04f34d-fuckyeah, portal: 192.168.1.10,3260]
Logout of [sid: 1, target: iqn.2001-05.com.equallogic:0-dead00-7cb4e0808-c4b000805b04f34d-fuckyeah, portal: 192.168.1.10,3260] successful.

Empezaremos viendo las interfaces que tenemos definidas:

# iscsiadm -m iface
default tcp,<empty>,<empty>,<empty>,<empty>
iser iser,<empty>,<empty>,<empty>,<empty>

Y añadiremos dos interfaces iSCSI que podemos llamar como las interfaces físicas:

# iscsiadm -m iface -I eth2 -o new
New interface eth0 added
# iscsiadm -m iface -I eth3 -o new
New interface eth1 added

Podemos ver como se han añadido:

# iscsiadm -m iface
default tcp,<empty>,<empty>,<empty>,<empty>
iser iser,<empty>,<empty>,<empty>,<empty>
eth2 tcp,<empty>,<empty>,<empty>,<empty>
eth3 tcp,<empty>,<empty>,<empty>,<empty>

Pero la interfaz no esta asociada a ninguna interfaz de red (iface.net_ifacename):

# iscsiadm -m iface -I eth2
# BEGIN RECORD 2.0-872.13.el5
iface.iscsi_ifacename = eth2
iface.net_ifacename = <empty>
iface.ipaddress = <empty>
iface.hwaddress = <empty>
iface.transport_name = tcp
iface.initiatorname = <empty>
iface.bootproto = <empty>
iface.subnet_mask = <empty>
iface.gateway = <empty>
iface.ipv6_autocfg = <empty>
iface.linklocal_autocfg = <empty>
iface.router_autocfg = <empty>
iface.ipv6_linklocal = <empty>
iface.ipv6_router = <empty>
iface.state = <empty>
iface.vlan_id = 0
iface.vlan_priority = 0
iface.vlan_state = <empty>
iface.iface_num = 0
iface.mtu = 0
iface.port = 0
# END RECORD

Por lo que deberemos modificar la interfaz iSCSI para asociarla a la interfaz de red:

# iscsiadm -m iface -I eth2 -n iface.net_ifacename -v eth2 -o update
eth2 updated.
# iscsiadm -m iface -I eth2
# BEGIN RECORD 2.0-872
iface.iscsi_ifacename = eth2
iface.net_ifacename = eth2
iface.ipaddress = <empty>
iface.hwaddress = <empty>
iface.transport_name = tcp
iface.initiatorname = <empty>
# END RECORD

Esto no podemos hacerlo con sesiones activas, ya que no se aplicarían los cambios:

# iscsiadm -m iface -I eth2 -o update -n iface.net_ifacename -v eth0
iscsiadm: Updating iface while iscsi sessions are using it. You must logout the running sessions then log back in for the new settings to take affect.
eth2 updated.

A continuación deberemos hacer el discovery de nuevo:

# iscsiadm -m discovery -t st -p 192.168.1.10
192.168.1.10:3260,1 iqn.2001-05.com.equallogic:0-dead00-7cb4e0808-c4b000805b04f34d-fuckyeah
192.168.1.10:3260,1 iqn.2001-05.com.equallogic:0-dead00-7cb4e0808-c4b000805b04f34d-fuckyeah

Hacemos login a los targets iSCSI:

# iscsiadm -m node -l 
Logging in to [iface: eth3, target: iqn.2001-05.com.equallogic:0-dead00-7cb4e0808-c4b000805b04f34d-fuckyeah, portal: 192.168.1.10,3260] (multiple)
Logging in to [iface: eth2, target: iqn.2001-05.com.equallogic:0-dead00-7cb4e0808-c4b000805b04f34d-fuckyeah, portal: 192.168.1.10,3260] (multiple)
Login to [iface: eth3, target: iqn.2001-05.com.equallogic:0-dead00-7cb4e0808-c4b000805b04f34d-fuckyeah, portal: 192.168.1.10,3260] successful.
Login to [iface: eth2, target: iqn.2001-05.com.equallogic:0-dead00-7cb4e0808-c4b000805b04f34d-fuckyeah, portal: 192.168.1.10,3260] successful.

Los cambios que hacemos mediante iscsadm se van guardando dentro de /var/lib/iscsi/:

# ls /var/lib/iscsi/
ifaces  isns  nodes  send_targets  slp  static
# cat /var/lib/iscsi/ifaces/eth
eth0  eth1  
# cat /var/lib/iscsi/ifaces/eth*
# BEGIN RECORD 2.0-872
iface.iscsi_ifacename = eth0
iface.transport_name = tcp
# END RECORD
# BEGIN RECORD 2.0-872
iface.iscsi_ifacename = eth1
iface.transport_name = tcp
# END RECORD

Seguiremos configurando multipath para que use los dos caminos que tenemos disponibles simétricamente. Mediante la opción -v3 podemos ver el resultado de la configuración actual:

# multipath -v3
hda: blacklisted
md0: blacklisted
ram0: blacklisted
ram10: blacklisted
ram11: blacklisted
ram12: blacklisted
ram13: blacklisted
ram14: blacklisted
ram15: blacklisted
ram1: blacklisted
ram2: blacklisted
ram3: blacklisted
ram4: blacklisted
ram5: blacklisted
ram6: blacklisted
ram7: blacklisted
ram8: blacklisted
ram9: blacklisted
sda: blacklisted
sdb: blacklisted
sdc: blacklisted
sdd: blacklisted
===== no paths =====

Quitamos la blacklist eliminando el fichero de configuración:

mv /etc/multipath.conf /etc/multipath.default

Y veremos como por defecto tenemos una configuración activo/pasivo:

# multipath -v2
# multipath -ll
36090a08880e0b47c4df3045b8000b0c4 dm-0 EQLOGIC,100E-00
[size=100G][features=0][hwhandler=0][rw]
\_ round-robin 0 [prio=1][active]
 \_ 9:0:0:0 sdc 8:32  [active][ready]
\_ round-robin 0 [prio=1][enabled]
 \_ 8:0:0:0 sdd 8:48  [active][ready]
[root@sunfire log]# 

Añadimos la siguiente configuración en /etc/multipath.conf:

blacklist {
        devnode "^(ram|raw|loop|fd|md|dm-|sr|scd|st)[0-9]*" # altres
        devnode "^hd[a-z][0-9]*" # cdrom
        devnode "^sd[ab][0-9]*"

        device {
                vendor "Dell"
                product "*"
        }
}

devices {
     device {
          vendor                  "EQLOGIC"
          product                 "100E-00"
          path_grouping_policy    multibus
	  getuid_callout 	  "/sbin/scsi_id -g -u -s /block/%n"
          features                "1 queue_if_no_path"
          path_checker            readsector0
          path_selector           "round-robin 0"
          failback                immediate
          rr_min_io               10
          rr_weight               priorities
     }
}

defaults {
        user_friendly_names yes
}

Tendremos ahora la configuración de multipath en activo/activo, con la opción -v2 veremos los cambios:

# multipath -v2
reload: mpath2 (36090a08880e0b47c4df3045b8000b0c4)  EQLOGIC,100E-00
[size=100G][features=1 queue_if_no_path][hwhandler=0][n/a]
\_ round-robin 0 [prio=2][undef]
 \_ 9:0:0:0 sdc 8:32  [active][ready]
 \_ 8:0:0:0 sdd 8:48  [active][ready]

Con multipath y la opción -ll veremos todos demás multipaths que tengamos en el equipo:

# multipath -v2
reload: mpath5 (36090a08880e0544c0af664f68f00e0c7)  EQLOGIC,100E-00
[size=600G][features=1 queue_if_no_path][hwhandler=0][n/a]
\_ round-robin 0 [prio=2][undef]
 \_ 12:0:0:0 sdi 8:128 [active][ready]
 \_ 11:0:0:0 sdj 8:144 [active][ready]
# multipath -ll
mpath2 (3600601605f211b00b89c77c4cfb0dd11) dm-1 DGC,RAID 1
[size=268G][features=1 queue_if_no_path][hwhandler=1 emc][rw]
\_ round-robin 0 [prio=1][active]
 \_ 0:0:1:1  sde 8:64  [active][ready]
\_ round-robin 0 [prio=0][enabled]
 \_ 0:0:0:1  sdb 8:16  [active][ready]
mpath5 (36090a08880e0544c0af664f68f00e0c7) dm-11 EQLOGIC,100E-00
[size=600G][features=1 queue_if_no_path][hwhandler=0][rw]
\_ round-robin 0 [prio=2][active]
 \_ 12:0:0:0 sdi 8:128 [active][ready]
 \_ 11:0:0:0 sdj 8:144 [active][ready]
mpath4 (3600601605f211b000852c529e962e111) dm-2 DGC,RAID 5
[size=50G][features=1 queue_if_no_path][hwhandler=1 emc][rw]
\_ round-robin 0 [prio=1][active]
 \_ 0:0:1:2  sdf 8:80  [active][ready]
\_ round-robin 0 [prio=0][enabled]
 \_ 0:0:0:2  sdc 8:32  [active][ready]
mpath3 (3600601605f211b00f626b616644fde11) dm-0 DGC,RAID 1
[size=268G][features=1 queue_if_no_path][hwhandler=1 emc][rw]
\_ round-robin 0 [prio=1][active]
 \_ 0:0:0:0  sda 8:0   [active][ready]
\_ round-robin 0 [prio=0][enabled]
 \_ 0:0:1:0  sdd 8:48  [active][ready]
[root@sparrow data]# 

En el caso que caiga una interfaz veremos en los logs lo siguiente:

Mar 13 19:28:42 pica kernel:  connection5:0: ping timeout of 5 secs expired, recv timeout 5, last rx 4306666988, last ping 4306671988, now 4306676988
Mar 13 19:28:42 pica kernel:  connection5:0: detected conn error (1011)
Mar 13 19:28:42 pica multipathd: sdd: readsector0 checker reports path is down 
Mar 13 19:28:42 pica multipathd: checker failed path 8:48 in map mpath2 
Mar 13 19:28:42 pica multipathd: mpath2: remaining active paths: 1 
Mar 13 19:28:42 pica multipathd: dm-0: add map (uevent) 
Mar 13 19:28:42 pica multipathd: dm-0: devmap already registered 
Mar 13 19:28:42 pica kernel: device-mapper: multipath: Failing path 8:48.
Mar 13 19:28:43 pica iscsid: Kernel reported iSCSI connection 5:0 error (1011) state (3)
Mar 13 19:28:45 pica iscsid: cannot make a connection to 192.168.1.10:3260 (-1,19)

En el momento que se recupere veremos:

Mar 13 19:30:02 pica kernel: bnx2 0000:05:00.1: eth2: using MSIX
Mar 13 19:30:02 pica kernel: ADDRCONF(NETDEV_UP): eth2: link is not ready
Mar 13 19:30:03 pica iscsid: cannot make a connection to 192.168.1.10:3260 (-1,19)
Mar 13 19:30:04 pica kernel: bnx2 0000:05:00.1: eth2: NIC Copper Link is Up, 1000 Mbps full duplex, receive & transmit flow control ON
Mar 13 19:30:04 pica kernel: ADDRCONF(NETDEV_CHANGE): eth2: link becomes ready
Mar 13 19:30:06 pica iscsid: cannot make a connection to 192.168.1.10:3260 (-1,19)
Mar 13 19:30:10 pica iscsid: connection5:0 is operational after recovery (29 attempts)
Mar 13 19:30:10 pica multipathd: sdd: readsector0 checker reports path is up 
Mar 13 19:30:10 pica multipathd: 8:48: reinstated 
Mar 13 19:30:10 pica multipathd: mpath2: remaining active paths: 2 
Mar 13 19:30:10 pica multipathd: dm-0: add map (uevent) 
Mar 13 19:30:10 pica multipathd: dm-0: devmap already registered 

Si tenemos gráficos de uso de la red iSCSI podremos comprobar que el tráfico se reparte entre las dos interfaces de red:

Tráfico iSCSI simetrico

Tráfico iSCSI simetrico

Deja un comentario:

XHTML - Tags permitidos:<a href="" title=""> <abbr title=""> <acronym title=""> <b> <blockquote cite=""> <cite> <code> <del datetime=""> <em> <i> <q cite=""> <s> <strike> <strong>