Hard Drive Hell
From KCLUG Wiki
Occasionally, I need to test a drive, to see if it "has problems". That can be subjective. I found myself doing the same things over and over. Badblocks and smart tests. I can't assume to be able to save some script file when I do this testing on a machine booted in to a ramdisk, so I needed something I could paste into a shell. I used to read the screen, and wait for the smart tests, and come back, but that was a pain. I used to look at the results, run the test again, but I'm lazy. So now I paste one line in, the machine does it's thing, and sends me the results as they are collected.
First, here it is all ugly, or in it's full glory, depending on your perspective:
drive=/dev/hdz ; passes=10 ; emailaddress=EmailAddress@GMail.com ; iter=0 ; while [[ ${iter} -lt ${passes} ]] ; do { echo "Beginning Iteration ${iter} on drive ${drive}" ; dd if=/dev/zero of=${drive} bs=1M; hdparm -y ${drive} ; hdparm -Y ${drive} ; hdparm -z ${drive} ; hdparm -t ${drive} ; hdparm -T ${drive} ; sleep $( echo "scale=0 ; $( /usr/sbin/smartctl -t short ${drive} | grep "Please wait" | grep "for test to complete" | sed -e 's/Please wait //' -e 's/ minutes for test to complete.//' -e 's/^[ \t]*//;s/[ \t]*$//' ) * 60 * 1.2" | bc ) ; /usr/sbin/smartctl -i -a -d ata ${drive} | grep -q "No Conveyance Self-test supported." ; if [[ $? -eq 1 ]]; then sleep $( echo "scale=0 ; $( /usr/sbin/smartctl -t conveyance ${drive} | grep "Please wait" | grep "for test to complete" | sed -e 's/Please wait //' -e 's/ minutes for test to complete.//' -e 's/^[ \t]*//;s/[ \t]*$//' ) * 60 * 1.2" | bc ) ; fi ; sleep $( echo "scale=0 ; $( /usr/sbin/smartctl -t long ${drive} | grep "Please wait" | grep "for test to complete" | sed -e 's/Please wait //' -e 's/ minutes for test to complete.//' -e 's/^[ \t]*//;s/[ \t]*$//' ) * 60 * 1.2" | bc ) ; sleep $( echo "scale=0 ; $( /usr/sbin/smartctl -t offline ${drive} | grep "Please wait" | grep "for test to complete" | sed -e 's/Please wait //' -e 's/ seconds for test to complete.//' -e 's/^[ \t]*//;s/[ \t]*$//' ) * 1.2" | bc ) ; smartctl -i -a -d ata ${drive} ; badblocks -t random -w -s -v ${drive} ; echo "`hostname` hdhell ${drive} iteration ${iter}" | dd of=${drive} bs=1 ; echo "." ; } | mail -s "`hostname` hdhell ${drive} iteration ${iter}" ${emailaddress} ; iter=$(( ${iter}+1 )) ; done
Now here it is cleaned up to be pasted in to a script.
#Set your settings here.
drive=/dev/hdz
passes=10
emailaddress=YourEmailAddressHere@GMail.com
#Leave iter alone.
iter=0
#It will run this mess as many times as you specify in passes.
while [[ ${iter} -lt ${passes} ]]
do
{
#Print a title banner for clarity, and to ensure that the first thing the mail
#command gets is normal text.
echo "Beginning Iteration ${iter} on drive ${drive}"
#Completely zero out the drive.
dd if=/dev/zero of=${drive} bs=1M
#Perform some hdparm magic:
hdparm -y ${drive} #Enter low power mode.
hdparm -Y ${drive} #Enter lowest power mode.
hdparm -z ${drive} #Re-read the partition table, which should be blank now.
hdparm -t ${drive} #Run the (inaccurate) hdparm test without disk cache.
hdparm -T ${drive} #Run the (inaccurate) hdparm test with disk cache.
#The point of this was to powercycle the drive as much as possible then load it heavily
#Sleep for a duration calculated by the program bc while the smart test runs.
#bc calculates the number of seconds it should sleep with a 20% margin for drive slowness.
#It gets the number of minutes from the grep'd and sed'd output of smartctl.
#And the particular smartctl command starts the test that we're waiting for.
#First we run the short test, which usualy takes one or two minutes.
sleep $( echo "scale=0 ; $( /usr/sbin/smartctl -t short ${drive} | \
grep "Please wait" | \
grep "for test to complete" | \
sed -e 's/Please wait //' \
-e 's/ minutes for test to complete.//' \
-e 's/^[ \t]*//;s/[ \t]*$//' ) * 60 * 1.2" | \
bc )
#Check to see if Conveyance test is supported. Some drives emit serious errors when they
#don't support Conveyance testing and you request it
/usr/sbin/smartctl -i -a -d ata ${drive} | \
grep -q "No Conveyance Self-test supported."
if [[ $? -eq 1 ]]
then
#Then we run the conveyance test. This usually takes two to six minutes.
sleep $( echo "scale=0 ; $( /usr/sbin/smartctl -t conveyance ${drive} | \
grep "Please wait" | \
grep "for test to complete" | \
sed -e 's/Please wait //' \
-e 's/ minutes for test to complete.//' \
-e 's/^[ \t]*//;s/[ \t]*$//' ) * 60 * 1.2" | \
bc )
fi
sleep $( echo "scale=0 ; $( /usr/sbin/smartctl -t conveyance ${drive} | \
grep "Please wait" | \
grep "for test to complete" | \
sed -e 's/Please wait //' \
-e 's/ minutes for test to complete.//' \
-e 's/^[ \t]*//;s/[ \t]*$//' ) * 60 * 1.2" | \
bc )
#Then we run the long test, which varies widely.
sleep $( echo "scale=0 ; $( /usr/sbin/smartctl -t long ${drive} | \
grep "Please wait" | \
grep "for test to complete" | \
sed -e 's/Please wait //' \
-e 's/ minutes for test to complete.//' \
-e 's/^[ \t]*//;s/[ \t]*$//' ) * 60 * 1.2" | \
bc )
#Then we run the offline test, which also varies.
sleep $( echo "scale=0 ; $( /usr/sbin/smartctl -t offline ${drive} | \
grep "Please wait" | \
grep "for test to complete" | sed -e 's/Please wait //' \
-e 's/ seconds for test to complete.//' \
-e 's/^[ \t]*//;s/[ \t]*$//' ) * 1.2" | \
bc )
#Display the current smart data, which will show the results of the tests.
smartctl -i -a -d ata ${drive}
#Run badblocks on the drive to check the drive at the block layer for trouble.
badblocks -t random -w -s -v ${drive}
#"Tag" the drive so you can tell later if its filled with random because it went through hehell as opposed to being encrypted.
echo "`hostname` hdhell ${drive} iteration ${iter}" | dd of=${drive} bs=1
#Echoing a dot makes the mail command work better when other errors in the output above show through.
echo "."
#Mail the output of everything between the curly braces to the admin with a clever subject.
} | mail -s "`hostname` hdhell ${drive} iteration ${iter}" ${emailaddress}
iter=$(( ${iter}+1 ))
done
You will get an email after every iteration completes, like this one:
Date: Thu, 15 Jan 2009 09:03:13 -0600 From: root <root@YourHostname.here> To: YourEmailAddressHere@GMail.com Subject: YourHostname.here hdhell /dev/hdi iteration 0 Beginning Iteration 0 on drive /dev/hdi /dev/hdi: issuing standby command /dev/hdi: issuing sleep command /dev/hdi: /dev/hdi: Timing buffered disk reads: 48 MB in 3.09 seconds = 15.55 MB/sec /dev/hdi: Timing cached reads: 324 MB in 2.02 seconds = 160.12 MB/sec smartctl version 5.36 [i686-redhat-linux-gnu] Copyright (C) 2002-6 Bruce Allen Home page is http://smartmontools.sourceforge.net/ === START OF INFORMATION SECTION === Model Family: Western Digital Caviar family Device Model: WDC WD400BB-75AUA1 Serial Number: WD-WMA6R3416648 Firmware Version: 18.20D18 User Capacity: 40,020,664,320 bytes Device is: In smartctl database [for details use: -P show] ATA Version is: 5 ATA Standard is: Exact ATA specification draft version not indicated Local Time is: Thu Jan 15 04:48:45 2009 CST SMART support is: Available - device has SMART capability. SMART support is: Enabled === START OF READ SMART DATA SECTION === SMART overall-health self-assessment test result: PASSED General SMART Values: Offline data collection status: (0x82) Offline data collection activity was completed without error. Auto Offline Data Collection: Enabled. Self-test execution status: ( 0) The previous self-test routine completed without error or no self-test has ever been run. Total time to complete Offline data collection: (2040) seconds. Offline data collection capabilities: (0x1b) SMART execute Offline immediate. Auto Offline data collection on/off support. Suspend Offline collection upon new command. Offline surface scan supported. Self-test supported. No Conveyance Self-test supported. No Selective Self-test supported. SMART capabilities: (0x0003) Saves SMART data before entering power-saving mode. Supports SMART auto save timer. Error logging capability: (0x01) Error logging supported. No General Purpose Logging support. Short self-test routine recommended polling time: ( 2) minutes. Extended self-test routine recommended polling time: ( 32) minutes. SMART Attributes Data Structure revision number: 16 Vendor Specific SMART Attributes with Thresholds: ID# ATTRIBUTE_NAME FLAG VALUE WORST THRESH TYPE UPDATED WHEN_FAILED RAW_VALUE 1 Raw_Read_Error_Rate 0x000b 200 200 051 Pre-fail Always - 0 3 Spin_Up_Time 0x0007 101 101 021 Pre-fail Always - 3766 4 Start_Stop_Count 0x0032 100 100 040 Old_age Always - 545 5 Reallocated_Sector_Ct 0x0032 200 200 112 Old_age Always - 0 7 Seek_Error_Rate 0x000b 200 200 051 Pre-fail Always - 0 9 Power_On_Hours 0x0032 084 084 000 Old_age Always - 11736 10 Spin_Retry_Count 0x0013 100 100 051 Pre-fail Always - 2 11 Calibration_Retry_Count 0x0013 100 100 051 Pre-fail Always - 0 12 Power_Cycle_Count 0x0032 100 100 000 Old_age Always - 269 196 Reallocated_Event_Count 0x0032 200 200 000 Old_age Always - 0 197 Current_Pending_Sector 0x0012 200 200 000 Old_age Always - 0 198 Offline_Uncorrectable 0x0012 200 200 000 Old_age Always - 0 199 UDMA_CRC_Error_Count 0x000a 200 253 000 Old_age Always - 0 200 Multi_Zone_Error_Rate 0x0009 200 146 051 Pre-fail Offline - 0 SMART Error Log Version: 1 No Errors Logged SMART Self-test log structure revision number 1 Num Test_Description Status Remaining LifeTime(hours) LBA_of_first_error # 1 Extended offline Completed without error 00% 812 - # 2 Short offline Completed without error 00% 812 - # 3 Extended offline Completed without error 00% 783 - # 4 Short offline Completed without error 00% 782 - # 5 Short offline Completed without error 00% 766 - Device does not support Selective Self Tests/Logging .
