[[TOC]]

For our example, we are configuring a HP 2810-48G switch.  These commands should also work with the HP5400zl series.

Keep in mind, this is an example configuration.  You site may:
 * Not require full remote access to your ESXi machine (no DRAC or iLO connection).
 * Use IPMI in place of Serial and Power controllers.

== Network Overview ==

There are a number of different networks and hosts involved with a traditional Emulab setup.

For servers, we will have three machines hosted on a VMWare server:

 * Boss   hosts the web interface, main testbed logic, and database.
 * Users  exports filesystems and acts as a general login machine.
 * Router routes between the various networks.

We also have on the network:
 * HP 2810 Ethernet Switch Management Interface
 * Dell Remote Access Card for VMWare server
 * VMWare management interface
 * Power Controller (APC7902)
 * Serial Controller (IBM X330 + Obsolete Cyclades multi-port serial card)



== Initial Ports ==

|| Port || Device ||
|| 1      || Uplink to Internet ||
|| 2      || Port for Dell Remote Access Card (DRAC) for the VMWare server (!PowerEdge 2950) ||
|| 3      || Trunked port for the VMWare installation on the !PowerEdge 2950 ||
|| 4      || APC 7902 Power Controller ||
|| 5      || Serial Controller  ||
|| 6-9    || Control Network ports for our testbed nodes ||

== VLANS ==

|| VLAN || IP Range || Use ||
|| 2002 || Depends (our minibed is behind a NAT on 10.0.23.0/24) || Internet VLAN ||
|| 2003 || 192.168.0.0/22 || Control Network VLAN ||
|| 2004 || 192.168.254.0/24 || Control Hardware VLAN ||
|| 2005 || 192.168.253.0/24 || Users Network ||
|| 2006 || 192.168.252.0/24 || Boss Network ||
|| 2007 || 192.168.224.0/20 || Node IPMI Network ||

== Diagram ==

[[Image(Mini DETER Network.png)]]

== Switch Configuration ==

Here is the configuration from the HP Procurve 2810 at the ISI mini testbed.  Please note that the max-vlans number can be, and should be, increased on switches that are capable of more (5400 series).  We use a trunked interface (port 3) to hook into our VMWare ESXi server.

{{{
hostname "hp1" 
max-vlans 256 
interface 1 
   name "UPLINK" 
exit
interface 2 
   name "VMWARE_DRAC" 
exit
interface 3 
   name "VMWARE" 
exit
interface 4 
   name "POWER_CONTROLLER" 
exit
interface 5 
   name "SERIAL_CONTROLLER" 
exit
ip default-gateway 192.168.254.254 
snmp-server community "public" Unrestricted 
snmp-server community "private" manager Unrestricted 
vlan 1 
   name "DEFAULT_VLAN" 
   untagged 13-48 
   no ip address 
   no untagged 1-12 
   exit 
vlan 2003 
   name "CONTROL" 
   untagged 6-12 
   tagged 3
   ip igmp
   exit 
vlan 2004 
   name "CONTROLHW" 
   untagged 4-5 
   ip address 192.168.254.1 255.255.255.0 
   tagged 3 
   exit 
vlan 2002 
   name "INTERNET" 
   untagged 1-2 
   tagged 3 
   exit 
vlan 2006 
   name "BOSS" 
   tagged 3
   ip igmp
   exit 
management-vlan 2004 
}}}

== VMWare Network Setup ==

For the minibed at ISI, the VMWare ESXi server is on a single trunked port.

[[Image(vmware_network.PNG)]]

Here is an example of using two NICs.  One NIC is connected to the Internet and the other is connected to the testbed switch.

[[Image(Two NIC ESXi.png)]]

=== Boss VM ===

 * em0 is on INTERNET
 * em1 is on BOSS

=== Users VM ===
 
 * em0 is on INTERNET
 * em1 is on USERS

=== Router VM ===

 * em0 is on USERS
 * em1 is on BOSS
 * em2 is on CONTROL
 * em3 is on CONTROLHW

=== HP Switch IGMP debugging ===

{{{
minibed# show ip igmp

 Status and Counters - IP Multicast (IGMP) Status

 VLAN ID : 1
 VLAN Name : DEFAULT_VLAN
  IGMP is not enabled

 VLAN ID : 2
 VLAN Name : _5
  IGMP is not enabled

 VLAN ID : 2003
 VLAN Name : CONTROL
 Querier Address : 192.168.1.254

  Active Group Addresses Reports Queries Querier Access Port
  ---------------------- ------- ------- -------------------
 
 VLAN ID : 2004
 VLAN Name : HWCONTROL
  IGMP is not enabled

 VLAN ID : 2005
 VLAN Name : USERS
  IGMP is not enabled

 VLAN ID : 2006
 VLAN Name : BOSS
 Querier Address : 192.168.252.254

  Active Group Addresses Reports Queries Querier Access Port
  ---------------------- ------- ------- -------------------
  234.5.6.30             29      29      48                 
 

minibed# 
}}}

If either the BOSS vlan or CONTROL vlan show up without IGMP support, you can enable it:

{{{
minibed# config
minibed(config)# vlan 2003
minibed(vlan-2003)# ip igmp
minibed(vlan-2003)# vlan 2006
minibed(vlan-2006)# ip igmp
minibed(vlan-2006)# exit
minibed(config)# exit
minibed# write memory
minibed# 
}}}

=== Setting vSwtich IGMP Version ===

I am not sure if this is necessary.  ESXi defaults to IGMP version 3.  We currently need IGMP version 2 support on our vSwitch.  The enable this, follow the following steps in the vSphere client:
                             
 * Click on the server
 * Click on the "Configuration Tab"
 * Click on "Advanced Settings" in the "Software" group
 * Select "Net" in the "Advanced Settings" dialog box tree
 * Set "Net.IGMPVersion" to "2"

[[Image(vSwitch IGMP.png)]]