RAID-1 failed, can't rebuild

Portal Home > Knowledgebase > Articles Database > RAID-1 failed, can't rebuild

Posted by Lem0nHead, 03-09-2010, 11:27 PM
hello I have a server running RAID-1 yesterday I got an E-mail from the driver service alerting that the RAID setup was in WARNING alert. When I check it, I found out Subunit drive 1 was OK, but Subunit drive 2 was "DEGRADED" The server was shutdown, the degraded harddrive was replaced and the rebuild started when it was close to 7%, it stoped with an ERROR (not "REBUILDING" or "WARNING" anymore) and the subunit drive 2 kept showing "DEGRADED" datacenter gave another try (by the way, I guess I should say it's 15minuteservers, and their tech Tim did a very good job) and it didn't finished rebuilding again (stopped near 5%) does anyone have experience with this situation? the tech tried to plug the original degraded HD on another computer and it also didn't work... so there was probably really a problem with that HD, not just with the controller but maybe the controller has a problem too, in which case replacing it might help... I just don't know how likely is that a new RAID card can damage my good HD Edit: apparently they're thinking the current (good) HD is having a problem, which is the reason the RAID can't rebuild although I don't know how likely it is, I just run a complete backup of the server and no errors was reported on dmesg, which would be expected if some sectors failed reading
Posted by jacksun, 03-10-2010, 12:38 AM
Image that drive now! I assume this is a hardware RAID1, in some cases with a drive fail in a RAID1 you must break the mirror first, swap the drive, then rebuild the mirror. I have seen this occur in both software and hardware RAID1 configs. The other thing I have had to do was document the RAID card settings, go into the BIOS on reboot, wipe the card config, reboot, reconfig the card EXACTLY the same, and reboot. Problem was the RAID card would not forget the config/problem after it happened and wouldn't realize it was a fresh drive. Be particular not to initialize the drives though, except the new one or you'll wipe your data! Image this drive now!
Posted by Lem0nHead, 03-10-2010, 12:47 AM
yes, I agree that imaging this drive is very important, but that means downtime and I don't know if the DC will do that job so I'm thinking my safest alternative (considering downtime) right now is to order a new server and move the customers from one server to the other
Posted by jacksun, 03-10-2010, 01:46 AM
Good plan, can they provision fast enough? Will they do the IP stuff and make it seamless for clients? DNS servers? Custom scripts? Try the good drive in a new box?
Posted by Lem0nHead, 03-10-2010, 01:49 AM
in fact I don't know any DCs which does that the plan is to get a new server, on another IP and then I move the accounts from this one to the new one (after reducing the DNS TTL) I think it can be done in a couple days with some scripting
Posted by net, 03-10-2010, 05:35 AM
You should also make sure that the replacement of the drive has the same model. It can cause a problem in rebuild also if the drive do not have the same model.
Posted by zapruder, 03-10-2010, 10:29 AM
If you can boot the system on the 1 good drive do so. Take a drive image backup. Then restore it onto a new set of RAID1 drives.

Add to Favourites Print this Article