A Case Study in ZFS Data Recovery
System: 6 Disk Dell RAID 0
Operating System: FreeNAS
File System: ZFS
Location: London / Essex
Outcome: Successful recovery
- We received a call from a customer with a crashed Dell 6 disk RAID 0 server. The server was configured using FreeNAS and running the ZFS file system. Zpools was unavailable and the customer told us their system was currently with another data recovery company who were unable to progress the recovery.
-
The PERC H700 RAID controller is very common in these systems.
The RAID controller was a PERC H700, running in RAID which was presenting a large volume to the FreeNAS system. This is the incorrect way to setup such as system because by not allowing FreeNAS to access the individual disks and utilise RAIDZ, it is unable to utilise the redundancy ZFS provides. If a hardware RAID controller is required it should be setup as HBA.
- The hard drives arrived with us: they were 5x2TB Seagate drives (Constellation) and 1xToshiba 2TB.
- All the drives were cloned, but one was found to have firmware issues which once fixed also showed the drive was suffering from bad sectors. This drive was transferred to specific hardware which was able to successfully read most of these bad sectors. Eventually an excellent clone was obtained, with only approx 1000 bad sectors not read.
- Once all the hard drives had been cloned they were re-introduced back into the system.
- Information from the PERC controller card showed 4 drives online, one foreign and one ready – the Dell had kicked out the failed drive and the whole RAID 0 had failed.
- We therefore copied and re-coded the Dell RAID information (located at the rear of every drive) back to the hard drives (by copying and modifying the RAID management info from the original failed drive because it had been untouched and was original).
- The drive was re-introduced to the system and the RAID 0 would not come back online.
- The system was then booted into FreeNAS, but although the 12TB disk could be seen, no volumes or pools could be located. Nothing we did in FreeNAS gave any access to the data.
- We then replaced the FreeNAS system which was installed onto a USB stick with a Ubuntu system.
-
Running command line actions in Ubuntu was crucial in getting this data recovered.
The installation of Ubuntu allowed us to use tools which could see and access the 12TB ZFS volume. It showed heavy data corruption and the data folders were inaccessible.
- We then installed a customer ZFS version for Ubuntu from a repository and executed various terminal commands to find and mount the ZFS partition. We then repaired the data corruption.
- The server’s data, now repaired and accessible, was recovered to an external drive.
> Links
Main RAID and Server Data Recovery Page
Main Dell RAID and Server Recovery Page