I had a SQL query that had to update various indexed unique fields in my database which went something like this:
UPDATE page SET path = REPLACE(path, 'junk', '') ON DUPLICATE KEY DELETE path
Obviously this isn’t a valid MySQL query. The work around is to use UPDATE IGNORE and then delete any rows after which contains the substring ‘junk’:
UPDATE IGNORE page SET path = REPLACE(path, 'junk', ''); DELETE FROM page WHERE path LIKE '%junk%'
After buying 2 new 2TB Western Digital Green hard drives and setting it up as part of a Linux software-based Raid 5, I noticed that the drives would ‘click’ every time a file is accessed after a period of inactivity. Running smartctl -a /dev/
The SMART report showed the following:
=== START OF INFORMATION SECTION === Device Model: WDC WD20EARS-00J99B0 Serial Number: WD-WCAWZ0840640 Firmware Version: 80.00A80 User Capacity: 2,000,398,934,016 bytes Device is: Not in smartctl database [for details use: -P showall] ATA Version is: 8 ATA Standard is: Exact ATA specification draft version not indicated Local Time is: Fri Sep 2 23:26:17 2011 MDT SMART support is: Available - device has SMART capability. SMART support is: Enabled === START OF READ SMART DATA SECTION === SMART overall-health self-assessment test result: PASSED General SMART Values: Offline data collection status: (0x80)Offline data collection activity was never started. Auto Offline Data Collection: Enabled. Self-test execution status: ( 0)The previous self-test routine completed without error or no self-test has ever been run. Total time to complete Offline data collection: (40200) seconds. Offline data collection capabilities: (0x7b) SMART execute Offline immediate. Auto Offline data collection on/off support. Suspend Offline collection upon new command. Offline surface scan supported. Self-test supported. Conveyance Self-test supported. Selective Self-test supported. SMART capabilities: (0x0003)Saves SMART data before entering power-saving mode. Supports SMART auto save timer. Error logging capability: (0x01)Error logging supported. General Purpose Logging supported. Short self-test routine recommended polling time: ( 2) minutes. Extended self-test routine recommended polling time: ( 255) minutes. Conveyance self-test routine recommended polling time: ( 5) minutes. SCT capabilities: (0x3035)SCT Status supported. SCT Feature Control supported. SCT Data Table supported. SMART Attributes Data Structure revision number: 16 Vendor Specific SMART Attributes with Thresholds: ID# ATTRIBUTE_NAME FLAG VALUE WORST THRESH TYPE UPDATED WHEN_FAILED RAW_VALUE 1 Raw_Read_Error_Rate 0x002f 200 200 051 Pre-fail Always - 0 3 Spin_Up_Time 0x0027 151 151 021 Pre-fail Always - 9441 4 Start_Stop_Count 0x0032 100 100 000 Old_age Always - 16 5 Reallocated_Sector_Ct 0x0033 200 200 140 Pre-fail Always - 0 7 Seek_Error_Rate 0x002e 200 200 000 Old_age Always - 0 9 Power_On_Hours 0x0032 100 100 000 Old_age Always - 72 10 Spin_Retry_Count 0x0032 100 253 000 Old_age Always - 0 11 Calibration_Retry_Count 0x0032 100 253 000 Old_age Always - 0 12 Power_Cycle_Count 0x0032 100 100 000 Old_age Always - 12 192 Power-Off_Retract_Count 0x0032 200 200 000 Old_age Always - 8 193 Load_Cycle_Count 0x0032 200 200 000 Old_age Always - 121 194 Temperature_Celsius 0x0022 125 121 000 Old_age Always - 27 196 Reallocated_Event_Count 0x0032 200 200 000 Old_age Always - 0 197 Current_Pending_Sector 0x0032 200 200 000 Old_age Always - 0 198 Offline_Uncorrectable 0x0030 100 253 000 Old_age Offline - 0 199 UDMA_CRC_Error_Count 0x0032 200 200 000 Old_age Always - 0 200 Multi_Zone_Error_Rate 0x0008 100 253 000 Old_age Offline - 0
Although the Load_Cycle_count isn’t that high, the drive was mostly idle or busy rebuilding the raid array which doesn’t justify the ~1.7 cycle counts per hour.
The solution to this is to run Western Digital’s WDIdle3 utility which allows a custom timeout to be set instead of the default 8 seconds. Unfortunately, WDidle3 is a DOS only program and therefore the computer must be booted in DOS before the program can be executed.
If you’re like me and don’t have DOS readily available and have no CDRom or a floopy drive, the easiest way is to load up FreeDOS under PXE with the WDIdle3 utility as part of the FreeDOS image.
Step 1: Prepare the fdboot.img image with the WDidle3 utility
mount -o loop -t vfat fdboot.img /mnt/freedos cp wdidle3.exe /mnt/freedos umount /mnt/freedos
Step 2: Configure PXE to boot the fdboot.img image as a ramdisk
vi /tftpboot/pxelinux.cfg/default # Then add the following entry: label FreeDOS kernel memdisk append initrd=fdboot.img
Step 3: Boot into PXE and boot into FreeDOS. A grub menu should appear:
Select #4 “FreeDOS Live CD with HIMEM only”
Step 4: Run the WDIdle3 utility with /S
* Yes, these screenshots were done under vmware. Couldn’t take a picture of it in action.
After disabling the timer in my WD20EARS, the Load_Cycle_Count remains virtually static and the ‘click’s have disappeared.
After installing lm_sensors and running sensors-detect, the following happens:
*omitting first part of sensors-detect output*
Now follows a summary of the probes I have just done.
Just press ENTER to continue:
Driver `w83627hf':
* ISA bus, address 0x290
Chip `Winbond W83637HF/HG Super IO Sensors' (confidence: 9)
Do you want to overwrite /etc/sysconfig/lm_sensors? (YES/no):
Starting lm_sensors: loading module w83627hf No sensors found!
Make sure you loaded all the kernel drivers you need.
Try sensors-detect to find out which these are.
[FAILED]
Unloading i2c-dev... OK
From the above, it is obvious that a sensor is found, but the module is not being loaded. Check dmesg to see if there is an ACPI conflict:
w83627hf: Found W83637HF chip at 0x290 ACPI: I/O resource w83627hf [0x295-0x296] conflicts with ACPI region IP__ [0x295-0x296] ACPI: If an ACPI driver is available for this device, you should use it instead of the native driver i2c /dev entries driver w83627hf: Found W83637HF chip at 0x290 ACPI: I/O resource w83627hf [0x295-0x296] conflicts with ACPI region IP__ [0x295-0x296] ACPI: If an ACPI driver is available for this device, you should use it instead of the native driver i2c /dev entries driver w83627hf: Found W83637HF chip at 0x290 ACPI: I/O resource w83627hf [0x295-0x296] conflicts with ACPI region IP__ [0x295-0x296] ACPI: If an ACPI driver is available for this device, you should use it instead of the native driver To resolve this issue, addacpi_enforce_resources=laxto the boot parameters:hiddenmenu title CentOS (2.6.32-71.el6.x86_64) root (hd0,0) kernel /vmlinuz-2.6.32-71.el6.x86_64 ro root=UUID=75da8d80-9b93-4ea1-8f3c-0ce53b3b4d5c rd_NO_LUKS rd_NO_LVM rd_NO_MD rd_NO_DM LANG=us SYSFONT=latarcyrheb-sun16 KEYBOARDTYPE=pc KEYTABLE=us crashkernel=auto acpi_enforce_resources=lax rhgb quiet initrd /initramfs-2.6.32-71.el6.x86_64.img
I have a spare Pentium 4 computer lying around. Unfortunately, it has no video card; nor did it have an onboard integrated graphics card. In order to install CentOS 6, I had to ‘borrow’ a card from another computer. While I had the card installed, I ensured the BIOS did not halt on any errors and it booted off PXE after HD. After testing the boot process a few times, I ‘returned’ the card to its rightful place – leaving the pentium 4 without a card and me hoping the computer still works.
Upon turning the computer on, the POST process would beep a morse code “long short short” signifying there is no graphics card and the computer would freeze (The numlock key on the keyboard froze). After unplugging the hard drive and hard resetting the computer, it would by default boot via PXE without a problem (I can see tftp and nfs traffic as it began booting ubuntu). This proved that the computer indeed was getting past POST without a problem.
After hitting my head against the wall for a few hours, it appears that grub needs to be set into console mode otherwise grub will cause the boot process to hang. To resolve this, the following changes were made in /boot/grub/menu.lst
default=0 timeout=5 #splashimage=(hd0,0)/grub/splash.xpm.gz serial --unit=0 --speed=9600 --parity=no --stop=1 terminal --timeout=2 serial
Note that the splash image is commented out and the boot mode is now set to use the serial port (which the computer doesn’t have)
Once this was set, the computer boots without a problem.
The following is the benchmark results of:
Pentium 4 @ 2.93GHz
MS-7091 Motherboard
1 GB memory
1 80GB IDE 7.2k HDD
cat /proc/cpuinfo
processor : 0
vendor_id : GenuineIntel
cpu family : 15
model : 4
model name : Intel(R) Pentium(R) 4 CPU 2.93GHz
stepping : 9
cpu MHz : 2933.260
cache size : 1024 KB
physical id : 0
siblings : 2
core id : 0
cpu cores : 1
apicid : 0
initial apicid : 0
fpu : yes
fpu_exception : yes
cpuid level : 5
wp : yes
flags : fpu vme de pse tsc msr pae mce cx8 apic mtrr pge mca cmov pat pse36 clflush dts acpi mmx
fxsr sse sse2 ss ht tm pbe syscall nx lm constant_tsc pebs bts pni dtes64 monitor ds_cpl tm2 cid cx16 xtpr
bogomips : 5851.95
clflush size : 64
cache_alignment : 128
address sizes : 36 bits physical, 48 bits virtual
power management:
free -m
total used free shared buffers cached
Mem: 997 716 280 0 32 509
-/+ buffers/cache: 174 822
Swap: 1023 0 1023
The benchmark results follows:
========================================================================
BYTE UNIX Benchmarks (Version 5.1.2)
System: server: GNU/Linux
OS: GNU/Linux -- 2.6.32-71.el6.x86_64 -- #1 SMP Fri May 20 03:51:51 BST 2011
Machine: x86_64 (x86_64)
Language: en_US.utf8 (charmap="UTF-8", collate="UTF-8")
CPU 0: Intel(R) Pentium(R) 4 CPU 2.93GHz (5851.9 bogomips)
Hyper-Threading, x86-64, MMX, Physical Address Ext, SYSCALL/SYSRET
CPU 1: Intel(R) Pentium(R) 4 CPU 2.93GHz (5851.0 bogomips)
Hyper-Threading, x86-64, MMX, Physical Address Ext, SYSCALL/SYSRET
09:54:59 up 58 min, 1 user, load average: 0.00, 0.01, 0.00; runlevel 3
------------------------------------------------------------------------
Benchmark Run: Sun Aug 28 2011 09:54:59 - 10:22:57
2 CPUs in system; running 1 parallel copy of tests
Dhrystone 2 using register variables 9237768.7 lps (10.0 s, 7 samples)
Double-Precision Whetstone 1526.3 MWIPS (9.8 s, 7 samples)
Execl Throughput 1453.5 lps (29.9 s, 2 samples)
File Copy 1024 bufsize 2000 maxblocks 260995.2 KBps (30.0 s, 2 samples)
File Copy 256 bufsize 500 maxblocks 80632.3 KBps (30.0 s, 2 samples)
File Copy 4096 bufsize 8000 maxblocks 557034.8 KBps (30.0 s, 2 samples)
Pipe Throughput 558617.5 lps (10.0 s, 7 samples)
Pipe-based Context Switching 47254.6 lps (10.0 s, 7 samples)
Process Creation 4953.0 lps (30.0 s, 2 samples)
Shell Scripts (1 concurrent) 2069.1 lpm (60.0 s, 2 samples)
Shell Scripts (8 concurrent) 310.8 lpm (60.0 s, 2 samples)
System Call Overhead 598237.8 lps (10.0 s, 7 samples)
System Benchmarks Index Values BASELINE RESULT INDEX
Dhrystone 2 using register variables 116700.0 9237768.7 791.6
Double-Precision Whetstone 55.0 1526.3 277.5
Execl Throughput 43.0 1453.5 338.0
File Copy 1024 bufsize 2000 maxblocks 3960.0 260995.2 659.1
File Copy 256 bufsize 500 maxblocks 1655.0 80632.3 487.2
File Copy 4096 bufsize 8000 maxblocks 5800.0 557034.8 960.4
Pipe Throughput 12440.0 558617.5 449.0
Pipe-based Context Switching 4000.0 47254.6 118.1
Process Creation 126.0 4953.0 393.1
Shell Scripts (1 concurrent) 42.4 2069.1 488.0
Shell Scripts (8 concurrent) 6.0 310.8 518.1
System Call Overhead 15000.0 598237.8 398.8
========
System Benchmarks Index Score 436.7
------------------------------------------------------------------------
Benchmark Run: Sun Aug 28 2011 10:22:57 - 10:51:06
2 CPUs in system; running 2 parallel copies of tests
Dhrystone 2 using register variables 9042652.5 lps (10.0 s, 7 samples)
Double-Precision Whetstone 1577.6 MWIPS (9.8 s, 7 samples)
Execl Throughput 1535.1 lps (29.9 s, 2 samples)
File Copy 1024 bufsize 2000 maxblocks 187110.7 KBps (30.0 s, 2 samples)
File Copy 256 bufsize 500 maxblocks 51088.8 KBps (30.0 s, 2 samples)
File Copy 4096 bufsize 8000 maxblocks 534814.5 KBps (30.0 s, 2 samples)
Pipe Throughput 540751.4 lps (10.0 s, 7 samples)
Pipe-based Context Switching 71007.1 lps (10.0 s, 7 samples)
Process Creation 6744.7 lps (30.0 s, 2 samples)
Shell Scripts (1 concurrent) 2378.1 lpm (60.0 s, 2 samples)
Shell Scripts (8 concurrent) 311.5 lpm (60.1 s, 2 samples)
System Call Overhead 808664.0 lps (10.0 s, 7 samples)
System Benchmarks Index Values BASELINE RESULT INDEX
Dhrystone 2 using register variables 116700.0 9042652.5 774.9
Double-Precision Whetstone 55.0 1577.6 286.8
Execl Throughput 43.0 1535.1 357.0
File Copy 1024 bufsize 2000 maxblocks 3960.0 187110.7 472.5
File Copy 256 bufsize 500 maxblocks 1655.0 51088.8 308.7
File Copy 4096 bufsize 8000 maxblocks 5800.0 534814.5 922.1
Pipe Throughput 12440.0 540751.4 434.7
Pipe-based Context Switching 4000.0 71007.1 177.5
Process Creation 126.0 6744.7 535.3
Shell Scripts (1 concurrent) 42.4 2378.1 560.9
Shell Scripts (8 concurrent) 6.0 311.5 519.2
System Call Overhead 15000.0 808664.0 539.1
========
System Benchmarks Index Score 450.1
Suppose I have a server at 10.1.1.5 with a samba share. In order to mount it automatically every time my machine boots up, I’ll need to have an entry in /etc/fstab:
//10.1.1.5/Volume_1 /mnt/box cifs credentials=/etc/cifspasswd 0 0
The credentials file contains:
username=leo
password=mypassword
The first time I ran mount -a, I got the following error:
root@server:~# mount -a
mount: wrong fs type, bad option, bad superblock on //box/Volume_1,
missing codepage or other error
In some cases useful info is found in syslog - try
dmesg | tail or so
with dmesg showing:
CIFS VFS: No username specified
CIFS VFS: cifs_mount failed w/return code = -22
To resolve this, install cifs-utils
yum -y install cifs-utils
Although there are scripts like DenyHosts, I wanted something that I could run without needing python. What I came up with is a simple one-liner shell command:
cat /var/log/secure*|grep Bye|awk '{print $9}'|sort|uniq --count | awk '$1 >= 50{print "sshd: " substr($2, 0, length($2)-1)}' >> /etc/hosts.deny
That should dump all the IPs in your /var/log/secure which failed at least 50 ssh authentication into your hosts.deny file.
One caveat with this is that multiple IPs will be added to the deny.hosts file if this is ran multiple times. However, this should be easily resolved by saving the IPs in a separate file, and then removing duplicate values using sort & uniq before being dumped into the hosts.deny file.
In order to mount a directory, image or device with a space in the file path such as:
/mnt/box/downloads/OS/Windows 7 SP1/en_windows_7_with_sp1_dvd.iso ==> /tftpboot/images/windows7
one can type in:
mount -t udf -o loop /mnt/box/downloads/OS/Windows\ 7\ SP1/en_windows_7_with_sp1_dvd.iso /tftpboot/images/windows7/
However, in order to add this entry to the fstab file, one must replace spaces with \040:
# cat /etc/fstab
/mnt/box/downloads/OS/Windows\0407\040SP1/en_windows_7_with_dvd.iso /tftpboot/images/windows7 udf ro,loop,auto 0 0
I was trying to setup VirtualBox on my Athlon 2600+ server under my desk to run Windows XP. The following are the steps which I used to install and use Virtual Box:
Install VirtualBox on CentOS 5
wget http://download.virtualbox.org/virtualbox/4.0.8/VirtualBox-4.0-4.0.8_71778_rhel5-1.i386.rpm yum -y install libGLU SDL rpm -ivh VirtualBox-4.0-4.0.8_71778_rhel5-1.i386.rpm
Setting up a VM
As per the documentation at: http://www.virtualbox.org/manual/ch07.html#idp11472496
VBoxManage createvm --name "WindowsXP" --ostype WindowsXP --register VBoxManage modifyvm "WindowsXP" --memory 256 --acpi on --boot1 dvd --nic1 bridged --bridgeadapter1 eth0 VBoxManage createhd --filename "WindowsXP.vdi" --size 10000 VBoxManage storagectl "WindowsXP" --name "IDE Controller" --add ide --controller PIIX4 VBoxManage storageattach "WindowsXP" --storagectl "IDE Controller" --port 0 --device 0 --type hdd --medium "WindowsXP.vdi" VBoxManage storageattach "WindowsXP" --storagectl "IDE Controller" --port 0 --device 1 --type dvddrive --medium /mnt/box/downloads/OS/windows_xp-sp3.iso VBoxManage modifyvm "WindowsXP" --vrde on
To start the newly created virtual machine, run:
VBoxHeadless --startvm "WindowsXP"
Making Remote Desktop Work
By default, the Virtual Box install does not come with the extension pack which enables VRDE. If you attempt to start the VM with VRDE enabled:
VBoxHeadless --startvm "WindowsXP" --vrde on
You will see this error in your log file:
VRDE: VirtualBox Remote Desktop Extension is not available.
To install the extension pack, download the latest extension pack at: http://download.virtualbox.org/virtualbox/
For example:
wget http://download.virtualbox.org/virtualbox/4.0.8/Oracle_VM_VirtualBox_Extension_Pack-4.0.8-71778.vbox-extpack VBoxManage extpack install Oracle_VM_VirtualBox_Extension_Pack-4.0.8-71778.vbox-extpack
First, unmount the image by running:
VBoxManage storageattach "WindowsXP" --storagectl "IDE Controller" --port 0 --device 1 --type dvddrive --medium emptydrive
then remount the drive as shown above.
One of my startup scripts (for sphinx) returned the following message on every startup on my XEN based VPS which prevented sphinx from ever starting on bootup:
sudo: sorry, you must have a tty to run sudo
To fix this issue, edit /etc/sudoers (using visudo) and edit out
Defaults requiretty