This is a good primer for getting familiar with using Infiniband with Redhat/Centos Linux.
http://people.redhat.com/dledford/infiniband_get_started.html
Getting Started with InfiniBand
The first step to using a new infiniband based network is to get the right packages installed. These are the infiniband related packages we ship and what they are there for (Note, the Fedora packages have not all been built or pushed to the repos yet, so their mention here is as a "Coming soon" variety, not an already done variety):
sernet.de maintain the latest Samba releases in a yum repository, allowing for an easy and painless install or upgrade of Samba on your yum based Linux distribution.
To install the latest available Samba execute the following commands at the shell:
# cd /etc/yum.repos.d # wget http://ftp.sernet.de/pub/samba/experimental/centos/5/sernet-samba.repo # yum install samba
To upgrade an existing Samba install:
# cd /etc/yum.repos.d # wget http://ftp.sernet.de/pub/samba/experimental/centos/5/sernet-samba.repo ## Note: edit sernet-samba.repo and add the line "gpgcheck=false" otherwise ## it will not install as it is not signed
# yum update samba
Note: These steps will install the very latest build available at sernet.de.
If you require a less bleeding edge version of Samba, use the "tested" repo. This can be found at the following URL: http://ftp.sernet.de/pub/samba/tested/rhel/5
If when your attempt to create device meta-data fails this is drbd preventing you from corrupting a file system present on the target partition.
$ drbdadm create-md drbd0
v08 Magic number not found
md_offset 30005817344
al_offset 30005784576
bm_offset 30004867072
Found ext2 filesystem which uses 190804004 kB
current configuration leaves usable 29301628 kB
Device size would be truncated, which
would corrupt data and result in
'access beyond end of device' errors.
You need to either
* use external meta data (recommended)
* shrink that filesystem first
* zero out the device (destroy the filesystem)
Operation refused.
Command 'drbdmeta /dev/drbd0 v08 /dev/sda4 internal create-md' terminated with exit code 40
drbdadm aborting
Once you have confirmed that the file system present on the target partition is no longer required at the prompt type the following:
Replace /dev/sdaX with the block device you are targeting.
dd if=/dev/zero of=/dev/sdaX bs=1M count=128
Once this has completed the drbdadm create-md drbd0 command will complete with a "success."
$ drbdadm create-md drbd0
v08 Magic number not found
v07 Magic number not found
v07 Magic number not found
v08 Magic number not found
Writing meta data...
initialising activity log
NOT initialized bitmap
New drbd meta data block successfully created.
success
$
VMware products, such as ESX, Workstation, Server, and Fusion, come with a built-in VNC server to access guests.
This allows you to connect to the guest without having a VNC server installed in the guest - useful if a server doesn't exist for the guest or if you need access some time when a server would not work (say during the boot process). It's also good in conjunction with Headless Mode.
The VNC server is set up on a per-VM basis, and is disabled by default. To enable it, add the following lines to the .vmx:
RemoteDisplay.vnc.enabled = "TRUE" RemoteDisplay.vnc.port = "5901"
You can set a password with RemoteDisplay.vnc.key; details for how to calculate the obfuscated value given a plaintext password are in Compute hashed password for use with RemoteDisplay.vnc.key.
If you want more than one VM set up in this manner, make sure they have unique port numbers. To connect, use a VNC client pointing at host-ip-address:port. If you connect from a different computer, you may have to open a hole in the OS X firewall. If you use Leopard's Screen Sharing.app on the same computer as Fusion, don't use port 5900 since Screen Sharing refuses to connect to that.
It would appear that gigabit LAN is not! In fact it often runs at the same speed as 100Mbps LAN. Let's look at why exactly.
After configuring your network you can use the ifconfig command to see what speeds the LAN is connected. Even though 1000Mbps is reported by the card, the reality is that the overall throughtput may well be ~100Mpbs. You can try copying a large file using scp to demonstrate this.
As it turns out, in order to use a gigabit LAN you need to use CAT6 cables. CAT5 and CAT5E are not good enough. End result, the ethernet cards throttle back the speed to reduce dropped packets and errors.
You can find a good article here titled Squeeze Your Gigabit NIC for Top Performance. After tuning up the TCP parameters i found that it made no dfifference. The principal reasons behind low gigabit ethernet performance can be summed up as follows.
- Need to use CAT6 cables
- Slow Disk speed
- Limitations of the PCI bus which the gigabit ethernet cards use
You can get an idea about the disk speed using the hdparm command:
Display the disk partitions and choose the main linux partition which has the / filesystem.
# fdisk -l
Then get disk cache and disk read statistics:
# hdparm -Tt /dev/sda0
On my desktop system the sata disk perfomance is a limiting factor. These were the results:
/dev/sda1:
Timing cached reads: 9984 MB in 2.00 seconds = 4996.41 MB/sec
Timing buffered disk reads: 84 MB in 3.13 seconds = 58.49 MB/sec
Well, that equates to a raw disk read speed of 58.49 * 8 = 467Mbps which is half the speed of a gigabit LAN.
So.. NAS storage with lots of memory looks to be the way to go... If you use the right cables!
When you start the loadbalancer.org appliance you will see the following:
Default login:
Username: root
Password: loadbalancer
Access to webclient from an external client is:
http://192.168.1.129:9080
http://192.168.1.129:9443
You can access the web administrator using the IP and ports described onscreen.
For the sri lanka porject we are looking for performance and the network diagram indicates we are happy to have the cluster on the same subnet as the rest of the network.
Direct routing is the fasted performance possible, it has the advantage over NAT that the Loadbalancer does not become a bottleneck for incoming and outgoing packets. With DR the loadbalancer simply examines incoming packets and the servers to route the packets directly back to the requesting user.
The web interfaceis the only way to fully configure the loadbalancer vm. The console tool lbwizard will get it initiallised and any further configurations can then be done via the webinterface.
Using lbwizard for the Sri lanka configuration follow these steps.
On the first Loadbalancer:
//Start
Is this unit part for a HA Pair?
YES
Have you already setup the Slave?
NO
Is this a one-armed configuration?
YES
Enter the IP Address for the interface eth0?
Enter IP address you wish to be assigned to the SLAVE loadbalancer.
Enter the netmask for interface eth0?
Enter netmask for the subnet.
Enter the Floating IP adrress?
Enter the IP address that will be IP assosiacted the the HA-pair of loadbalancers.
//Finish
On the 2nd loadbalancer VM, run the lbwizard.
//Start
Is this unit part of an HA-Pair?
YES
Have you already set up the Slave?
YES
What is the slave units UP address?
Enter the IP which you entered when configuring the other loadbalancer VM.
Is this a one-armed configuration?
YES
Enter the IP Address for the interface eth0?
Enter the IP that will be assigned to the MASTER loadbalancer
Enter the netmask for interface eth0?
Enter the subnet netmask.
Enter the Floating IP address?
Enter the IP address that will be IP assosiacted the the HA-pair of loadbalancers.
Enter the address of the default gateway?
Enter the deafult gateway for the subnet.
Enter the IP of the nameserver?
Enter the dns server.
Enter the port for the first Virtual server?
Enter 22 for ssh
Enter the IP address of the first real server?
Enter the real IP of the first appserver
//Finish
Now this is complete we need to go to the web admin interface to configure the 2nd Real Server. As the lbwizard program will only allow you to configure 1 real server.
Now login to the web admin using the default password:
username: loadbalancer
password: loadbalancer
Note: Connect to the IP you have now set for your master loadbalancer
Goto the edit configuration tab
Now click add a real server:
Enter a label
IP address of the server plus the port of the service i.e. 192.168.1.125:22
Edit Configuration -> Virtual Servers
persistancte -> NO
Scheduler-> LC
LC - Least-Connection: assign more jobs to real servers with
fewer active jobs.
Service to check -> custom1
Check port -> 22
Forwarding Method -> DR
Feedback Method -> Agent
Arp Problem when using DR
Every real server must be configured to respond to the VIP address as well as the RIP
address.
You can use iptables (netfilter) on the real server to re-direct incoming packets destined for the virtual
server IP address.
This is a simple case of adding the following command to your start up script (rc.local):
//replace 10.0.0.21 with the Virtual Server IP
iptables -t nat -A PREROUTING -p tcp -d 10.0.0.21 -j REDIRECT
chkconfig iptables on
Unfortunately java does not support __FILE__ and __LINE__ but you can get the same functionality with this code which can be placed in one of your libraries.
public static void showTrace(String msg) { if (msg.length() > 0) System.out.println(msg); System.out.println("Trace: " + "file " + new Throwable().getStackTrace()[1].getFileName() + " class " + new Throwable().getStackTrace()[1].getClassName() + " method " + new Throwable().getStackTrace()[1].getMethodName() + " line " + new Throwable().getStackTrace()[1].getLineNumber()); }
http://kbala.com/ie-9-supports-corner-radius/
Opening SSH to the outside world is a security risk. Here is how to restrict SSH access to certain IP addresses on a machine.
- Edit the /etc/hosts.allow file to include these lines, assuming your machine is on the 192.168.2.x nonrouting IP block, and you want to enable an external address of 217.40.111.121 IP block: Remember to add the period on the end of each incomplete IP number. If you have another complete IP address or range, add a space and that range on the end.
sshd,sshdfwd-X11: 192.168.2. 217.40.111.121
- Edit your /etc/hosts.deny file to include this line:
sshd,sshdfwd-X11:ALL
- These lines refuse SSH connections from anyone not in the IP address blocks listed.
Additionally you can restrict SSH access by username.
- Edit the /etc/ssh/sshd_config file and add the following lines
PermitRootLogin no
AllowUsers user1 user2 user3 etc
PasswordAuthentication yes
Now restart the ssh daemon for these changes to take effect
service sshd restart