On 12/16/23 15:38, Hendrik Boom via Dng wrote:
> I currently have no email access through my usual email address; thus
> I am resorting to gmail.
> The server was working fine last night, until:
>
> Last night my server became completely nonresponsive.  It was
> inaccessible through wifi, and it wouldn't respond to keyboard input. 
> Its screen remained black.
> But it seemed to be busy, judging from its blinking hard drive light.
>
> I rebooted it by the power button.  This was a hard reset.
>
> Subsequently it refused to boot.  It stalled in the initrd, claimint
> that I eeded to manually fsck the root partition.
>
> So I entered the appropriate fsck command from the keyboard, checking
> it twice, and leaving out the -y so that in case I made a mistake it
> shouldn't screw anything worse than it was already.  (If successful I
> planned to redo it with the -y)
>
> But fsck reported it could not find the partitoin it was to check.
>
> It was an lvm partion on a RAID.  (fsck can handle that, right?  Or
> did I do it wrong?  Something like
>     fsck /def/dm-1/VG1-long-name
> into the (initramfs) prompt.
> )

You need to start your RAID and lvm first, fsck cannot handle that for you.

You first need to check mdadm, you can do that with:

cat /proc/mdstat

if your devices are online they should show as "active sync".

if the mirror devices are not assembled, you might run this:

mdadm --assemble --scan --verbose

this would look into your partitions and enable your raid again. Check
again with cat /proc/mdstat, it will probably be syncing now.

once you have your RAID back, you should enable lvm volumes:

vgchange -ay

this should make your partitions available, and then you can run fsck if
needed, you should start with:

fsck -p /dev/dm1/VG1-long-name

An option to all these is to boot the devuan installer, or any linux
with a livedisk, and run it as a repair disk, it should find your RAID
and LVM and enable them, so you can check the disks.


>
> I also did an ls on that partition. That worked, except the top-level
> directory listing was gibberish.
>
> At this point I figured the root partition wa throroughly borked.
>  that was probably what the server was busy with when it went
> unresponsive -- borking the root partition.
>
> Any advice at this point?
>
> Next I figure it's time to wipe that partition, create a new one in
> that place or elsewhere, and and restore it from backup.  Yes, I have
> a recent backup! (unusual for this kind of question)
>
> But without booting, that won't work,
>
> * Approach 1:  Get a copy of refracta, and boot from that.
>
> So I downloaded
> https://get.refracta.org/files/beowulf/refracta10.6_xfce_amd64-20211226_1733.iso
> and dd'd it to /dev/sdc1 .  (Is that the way to do it?)
> before using it on the server, I decide to try booting from it on my
> laptop, just to rule out one thing that went wrong.
>
> (yes, I used the beowulf version to reduce any incompatibilities that
> might arise between different releases of Devuan, not that I expected any)
>
> I told the laptop to boot from USB, booted, and it complained it could
> not find any boot medium.  Evidently not copied correctly, or wrong
> kind of boot record.
>
> (my laptop and server are both, as far as I've ever known, BIOS machines).
>
> Any further advice?
>
> I have no website, and no access to my email system until this is
> resolved.
>
> * Approach 2:
>
> (not tried yet)
>
> Remove the relevant hard drive from the server and connect it to my
> laptop with as USB/SATA  adapter and mess with it that way.
> I'll have to activate the RAID on my laptop (how?) to process that
> drive properly.  Should be OK, since the RAID is currently defective,
> and there's another drive on my table that hasn't been activated yet,
> and it's a defective RAID with only one drive.
>
> Any other ideas?
>
> -- hendrik
>
> _______________________________________________
> Dng mailing list
> Dng@???
> https://mailinglists.dyne.org/cgi-bin/mailman/listinfo/dng

--
Hector Gonzalez
cacho@???