Sunday, 14 April 2013

NIC Bonding in RHEL

Steps for configuring bonding
In this document we are configuring bond0 with interfaces eth0 and eth1

Step 1- Load Kernel module
For a channel bonding interface to be valid, the kernel module must be loaded. To ensure that the module is loaded when the channel bonding interface is brought up, create a new file as root named <bonding>.conf in the /etc/modprobe.d/ directory. Note that we can name this file anything but it should with ends with a .conf extension. Insert the following line in this new file alias bond<N> bonding
Replace <N> with the interface number, such as 0. If we want to configure  configuring more than on bonding interface, For  each configured channel bonding interface, there must be a corresponding entry in  /etc/modprobe.d/<bonding>.conf file
In this example we are configuring bond0 and  file name is bonding.conf
[root@praji2 modprobe.d]# cat /etc/modprobe.d/bonding.conf
  alias bond0 bonding

Step2- create channel bonding interface
We  need to create a channel bonding interface configuration file on/etc/sysconfig/network-scripts/ directory called ifcfg-bond<N> ,replacing <N> with the number for the interface, such as 0 and specify the bonding parameters on the file. Here we are creating ifcfg-bond0 file with following contents
[root@praji2 network-scripts]# cat ifcfg-bond0
BONDING_OPTS="mode=0 miimon=1000"

Step 3- Configure Network interfaces
After the channel bonding interface is created, the network interfaces to be bound together must be configured by adding the MASTER= and SLAVE= directives to their configuration files. The configuration files for each of the channel-bonded interfaces can be nearly identical. For example, if two Ethernet interfaces are being channel bonded, both eth0 and eth1 may look like the following example
Interface eth0 configuration
 [root@praji2 network-scripts]# cat ifcfg-eth0
Interface eth1 configuration
[root@praji2 network-scripts]# cat ifcfg-eth1
After configuring the interfaces we have to bring up the bond by using command
[root@praji2 network-scripts]# ifconfig bond0 up
If the bonding is correctly configured we can view the configuration using ifconfig command
[root@praji2 network-scripts]# ifconfig
bond0     Link encap:Ethernet  HWaddr 00:0C:29:69:31:C4
          inet addr:  Bcast:  Mask:
          inet6 addr: fe80::20c:29ff:fe69:31c4/64 Scope:Link
          RX packets:19676 errors:0 dropped:0 overruns:0 frame:0
          TX packets:342 errors:0 dropped:0 overruns:0 carrier:0
          collisions:0 txqueuelen:0
          RX bytes:1623240 (1.5 MiB)  TX bytes:42250 (41.2 KiB)

eth0      Link encap:Ethernet  HWaddr 00:0C:29:69:31:C4
          RX packets:10057 errors:0 dropped:0 overruns:0 frame:0
          TX packets:171 errors:0 dropped:0 overruns:0 carrier:0
          collisions:0 txqueuelen:1000
          RX bytes:832257 (812.7 KiB)  TX bytes:22751 (22.2 KiB)
          Interrupt:19 Base address:0x2000

eth1      Link encap:Ethernet  HWaddr 00:0C:29:69:31:C4
          RX packets:9620 errors:0 dropped:0 overruns:0 frame:0
          TX packets:173 errors:0 dropped:0 overruns:0 carrier:0
          collisions:0 txqueuelen:1000
          RX bytes:791043 (772.5 KiB)  TX bytes:20207 (19.7 KiB)
          Interrupt:19 Base address:0x2080

lo        Link encap:Local Loopback
          inet addr:  Mask:
          inet6 addr: ::1/128 Scope:Host
          UP LOOPBACK RUNNING  MTU:16436  Metric:1
          RX packets:2 errors:0 dropped:0 overruns:0 frame:0
          TX packets:2 errors:0 dropped:0 overruns:0 carrier:0
          collisions:0 txqueuelen:0
          RX bytes:104 (104.0 b)  TX bytes:104 (104.0 b)

To view all existing bonds we can run following command, it will list bond0
[root@praji2 network-scripts]# cat /sys/class/net/bonding_masters
To view the existing mode of bonding we can use following command
[root@praji2 network-scripts]# cat /sys/class/net/bond0/bonding/mode
balance-rr 0
For verifying bonding , we can use following command. It will list bonding details
[root@praji2 network-scripts]# cat /proc/net/bonding/bond0
Ethernet Channel Bonding Driver: v3.5.0 (November 4, 2008)

Bonding Mode: load balancing (round-robin)
MII Status: up
MII Polling Interval (ms): 1000
Up Delay (ms): 0
Down Delay (ms): 0

Slave Interface: eth0
MII Status: up
Link Failure Count: 0
Permanent HW addr: 00:0c:29:69:31:c4

Slave Interface: eth1
MII Status: up
Link Failure Count: 0
Permanent HW addr: 00:0c:29:69:31:ce

Saturday, 30 March 2013

Cheat Sheet for VXDMP

VXDMP cheat Sheet.

list controllers vxdmpadm listctlr all
list enclosures vxdmpadm listenclosure all
vxdmpadm listenclosure <enclosure>
Display Paths vxdmpadm getsubpaths ctlr=fscsi2
vxdmpadm getsubpaths dmpnodename=<dmp-device-name>
Who controls the path vxdmpadm getdmpnode nodename=c3t2d0
vxdmpadm getdmpnode enclosure=<enclosure>
Controller (enable/disable) vxdmpadm disable ctlr=<controller>
vxdmpadm enable ctlr=<controller>
Statistics vxdmpadm iostat start
vxdmpadm iostat reset
vxdmpadm iostat show all
I/O balancing & load vxdmpadm getattr enclosure <enclosure> iopolicy
vxdmpadm setattr enclosure <enclosure> iopolicy=<policy>
balanced (default)
Path Type vxdmpadm setattr path <path-name> pathtype=<path-type>

Displays the restore/error daemon vxdmpadm stat restored
vxdmpadm stat errord
Starts the restore daemon vxdmpadm start restored
Stops the restore daemon vxdmpadm stop restored

Friday, 22 March 2013


Why Change scsi-initiator-id

Shared storage in clusters uses the Multi-Initiator capability of the SCSI
specification.  PCI systems use PCI host adapter cards for the SCSI interface.
Otherwise, the operation is the same as in SBus systems.  The scsi-initiator-id
must be changed because you cannot have two controllers (or SCSI devices) with
the same SCSI ID on the same SCSI chain.  This is true for all shared storage.

However, the scsi-initiator-id of controllers that link private storage should
be returned to 7 or it will conflict with the SCSI ID 6 of the CD-ROM.  Also,
for 6-slot MultiPacks you must change the SCSI IDs to the 9-14 range.  Use the
switch on the back of the MultiPack to change this.

The normal documented procedure for setting up dual-hosted MultiPacks is to set
the scsi-initiator-id for one node to 6 and then reset specific SCSI adapters 
(the ones that are not attached to the dual-hosted disks) to 7.  The 
scsi-initiator-id for the other node attached to the disk must be left at the
default setting (7). This approach may be somewhat ERROR-PRONE because you can
get SCSI errors if you do not reset ALL of the adapters that are NOT attached to
the dual-hosted disks to 7.

A method that works well is to leave the scsi-initiator-id for both nodes at 7.
Then, set the scsi-initiator-id for ONLY the adapters that are connected to the
dual-hosted disk to 6 on one of the nodes (normally the second node). 

Which scsi-initiator-id to Change and What to Set It To

Leave the scsi-initiator-id of one node set to the default (7) and change the
scsi-initiator-id to 6 for the other node. Do NOT change jumper settings on
any SCSI device (CD-ROM).

CAUTION:  Do NOT change the scsi-initiator-id to 8 because it WILL cause a
conflict with some other storage devices (D1000).

When to Change scsi-initiator-id

You must change the scsi-initiator-id BEFORE connecting the shared storage.
If the storage has already been connected, disconnect it first.

How to Change the scsi-initiator-id

Only change the scsi-initiator-id on one node in the chain of the dual-hosted
SCSI device. At the ok (OBP) prompt, use the probe-scsi-all command to
identify the controllers connected to shared storage and those connected to
private storage. You have to first set auto-boot to false then reset-all before
the probe-scsi-all command will work. Depending on your configuration, there are
two methods of doing this.

NOTE:  Use Method 1 if your system is an E450.

First, identify the SCSI adapters by entering the following from the boot PROM:

      ok  setenv auto-boot? false
      ok  reset-all
      ok  probe-scsi-all
      Target 2
          Unit 0        Disk    SEAGATE ST32171W SUN2.1G7462
      Target 3
          Unit 0        Disk    SEAGATE ST32171W SUN2.1G7462
      Target 2
          Unit 0        Disk    SEAGATE ST32171W SUN2.1G7462
      Target 3
          Unit 0        Disk    SEAGATE ST32171W SUN2.1G7462

Method 1

If more controllers are connected to the private storage than controllers
connected to shared storage, more scsi-initiator-id 7s than scsi-initiator-id 6s,
or more private storage controllers than shared storage controllers:

NOTE:  Steps 1 through 5 should be done on one of the nodes attached to the disk.
       Step 6 needs to be done on both nodes (assuming you are not using Ultra SCSI).

1.  Edit or create the nvramrc to set the scsi-initiator-id to 6 for these devices.
    From the OBP enter:
      ok  nvedit
      0:  probe-all install-console banner
      1:  cd /pci@6,4000/scsi@3
      2:  6 " scsi-initiator-id" integer-property
      3:  device-end
      4:  cd /pci@6,4000/scsi@2,1
      5:  6 " scsi-initiator-id" integer-property
      6:  device-end
      7:  banner (Control C)
2.  Do a ctrl-c, and store the nvramrc:

      ok  nvstore
3.  Set the system to use the nvramrc and reset auto-boot:

      ok  setenv use-nvramrc? true
      ok  setenv auto-boot? true
4.  Do a reset:

      ok  reset-all

5.  Edit the /etc/system file (on both nodes) and add the following line to set
    fast/wide SCSI (disable Ultra SCSI):
      set scsi_options=0x3f8
6.  Boot both systems and verify that you can see the multi-hosted disks from
    both nodes.

Method 2

If your system is not an E450, and there are more controllers connected to
shared storage than controller connected to private storage, more scsi-initiator-id6s
than scsi-initiator-id 7s, or more shared storage controllers than private
storage controllers, then:

1.  Set the global scsi-initiator-id to 6:

      ok  setenv scsi-initiator-id 6
      scsi-initiator-id = 6
2.  Edit or create the nvramrc script and set the scsi-initiator-id of the 
    controllers connected to private storage to 7.  The line numbers (0:, 1:,
    and so on) are printed by the OBP, for example:
CAUTION!  Insert EXACTLY one space after the double quote and before

      ok  nvedit
      0:  probe-all
      1:  cd /sbus@70,0/SUNW,fas@1,8800000
      2:  7 encode-int " scsi-initiator-id" property
      3:  device-end
      4:  cd /sbus@70,0/QLGC,isp@0,10000
      5:  7 encode-int " scsi-initiator-id" property
      6:  device-end
      7:  cd /sbus@50,0/SUNW,fas@1,8800000
      8:  7 encode-int " scsi-initiator-id" property
      9:  device-end
      10:  install-console
      11:  banner (Control C)
In this example you have set three controller scsi-initiator-ids to 7. Your script
may be different because you will be resetting controllers that were listed by the probe-scsi-all command.

The following is an example of the internal/external controllers in an E250/E450:

      ok  setenv auto-boot? false
      ok  reset-all
      ok  probe-scsi-all
      /pci@1f,4000/scsi@3 (internal controller)
      /pci@1f,4000/pci@4/SUNW,isptwo@4 (external controller)
      /pci@1f,4000/scsi@5 (external controller)
3.  Store or discard the changes.

    The changes you make through the nvedit command are done on a temporary
    copy of the nvramrc script. You can continue to edit this copy without risk.
    Once you have completed your edits, save the changes. If you are not sure about
    the changes, discard them.
    To store the changes, enter:
      ok  nvstore
    To discard the changes, enter:
      ok  nvquit
4.  Enter the nvedit command to create and store an nvramrc script, also, reset
    auto-boot back to the default of true.

      ok  setenv use-nvramrc? true
      ok  setenv auto-boot? true
5.  Connect the shared storage devices and then enter:

      ok  boot -r                                            

Monday, 11 March 2013

Cluster outlook

Hi Guys,
The image gives u complete outlook of any cluster technology


Tuesday, 12 February 2013

Renaming a disk group of VXVM

Hi Guys,
       Here i post steps to rename a disk group in VXVM.

Note: Steps remain same for any flavor


Step 1: Perform the pre-checks using below commands.

  1. vxprint -g <dg name> -hvpst
  2. vxdg list
  3. vxconfigd status
Step 2: Check if there is any other node which recognizes the diskgroups. Usually renaming of  
             DG is  needed on Clustered environment.

Step3 : Deport the DG to other node with below command.
  1. vxdg deport <diskgroup>
Step 4: Import the disk group.
  1. vxdg [-t] -n newdg import diskgroup
Assuming newdg is desired name to change.
diskgroup :--- old name.
-t --: Temporary (optional in this case.)

Step 5: Start the volumes using vxvol command.

  1. Vxvol -g <newdg name> startall