# Both upgrade from Deb 10->11 and new installs of Deb 11 fail. # Figure out I can still boot Deb 11 with 4.19 kernel, so kernel regression. # I'll select a kernel to boot, it says loading vmlinuz, then it shows "_" on a black screen forever. # On pressing enter it says it can't find boot media. # AMD Ryzen 3600, ASRock B450M Pro4, ATI Radeon 5770, gpt formatted MBR boot (no boot partition, /boot/ is on OS partition) # Attempt to boot after selection of kernel in grub2. Failure to find bootable partition, over and over and over with new kernels. # After compiling narrowed down the bad commit to somewhere between 5.5.19 and 5.6.7 # I'm pretty sure it has something to do with booting off of a grub block-list instead of from a proper boot partition. # This is my attempts versus time to figure out which commit ID introduced # the bug that prevents me from booting. By using Deb snapshots I figured out # it was between 5.5.0 and 5.6.7 somewhere. But git bisect was broken if starting # from v5.5.0 (but not v5.5.1) so I don't use bisect and just keep track of the binary search myself. # Using https://git.kernel.org/pub/scm/linux/kernel/git/stable/linux.git/refs/tags for the big jumps # and https://git.kernel.org/pub/scm/linux/kernel/git/stable/linux.git/log/?ofs=88000 for the little. git clone https://git.kernel.org/pub/scm/linux/kernel/git/stable/linux.git/ cd linux superkuh@janus:~/app_installs/linux/linux$ git bisect start superkuh@janus:~/app_installs/linux/linux$ git bisect good v5.5 superkuh@janus:~/app_installs/linux/linux$ git bisect bad v5.6.7 44e8b94aa090f158896b77c892166957cda72f7d was both good and bad rm -rf linux/ git clone https://git.kernel.org/pub/scm/linux/kernel/git/stable/linux.git/ cd linux superkuh@janus:~/app_installs/linux/linux$ git bisect reset superkuh@janus:~/app_installs/linux/linux$ git bisect start superkuh@janus:~/app_installs/linux/linux$ git bisect good v5.5 superkuh@janus:~/app_installs/linux/linux$ git bisect bad v5.6.7 44e8b94aa090f158896b77c892166957cda72f7d was both good and bad 2058 cd .. 2059 l 2060 rm -rf linux/ 2061 git clone https://git.kernel.org/pub/scm/linux/kernel/git/stable/linux.git/ 2062 l 2063 cd linux/ 2064 l 2065 git bisect start 2066 git bisect good v5.5 2067 git bisect bad v5.6.7 44e8b94aa090f158896b77c892166957cda72f7d was both good and bad 2068 git bisect reset 2069 git bisect good v5.5 2070 git bisect bad v5.6.7 44e8b94aa090f158896b77c892166957cda72f7d was both good and bad # Decide to screw bisect and do it live. Arbitrarily pick 5.5.10 as it's in the middle. # Note from much later: btw, I noticed the "good and bad" thing only happens with v5.5, not v5.5.1 # Note from much much later: the whole fakeroot problem was just trying it instead of using the ENV $fakeroot which was null because not needed. git checkout 7ee76f1601f39ab3941c8b1c9a19dfc58f7cea47 2102 cp /boot/config-$(uname -r) .config 2103 scripts/config --enable LOCALVERSION_AUTO 2104 scripts/config --disable DEBUG_INFO 2105 scripts/config --disable SYSTEM_TRUSTED_KEYRING 2106 scripts/config --set-str SYSTEM_TRUSTED_KEYS '' 2107 make olddefconfig 2108 fakeroot=$(grep -q 'Rules-Requires-Root: no' scripts/package/mkdebian || grep -q fakeroot scripts/Makefile.package scripts/package/Makefile || echo fakeroot) 2109 fakeroot make -j12 bindeb-pkg ... fakeroot errors here 2112 make -j12 bindeb-pkg 2092 git clone https://git.kernel.org/pub/scm/linux/kernel/git/stable/linux.git/ 2093 l 2094 cp linux/ linux-backup 2095 cp -r linux/ linux-backup 2096 l 2097 cd linux 2098 git show d271ab29230b1d0ceb426f374c221c4eb2c91c64 2099 apt search linux-image 2100 l 2101 git checkout 7ee76f1601f39ab3941c8b1c9a19dfc58f7cea47 2102 cp /boot/config-$(uname -r) .config 2103 scripts/config --enable LOCALVERSION_AUTO 2104 scripts/config --disable DEBUG_INFO 2105 scripts/config --disable SYSTEM_TRUSTED_KEYRING 2106 scripts/config --set-str SYSTEM_TRUSTED_KEYS '' 2107 make olddefconfig 2108 fakeroot=$(grep -q 'Rules-Requires-Root: no' scripts/package/mkdebian || grep -q fakeroot scripts/Makefile.package scripts/package/Makefile || echo fakeroot) 2109 fakeroot make -j12 bindeb-pkg 2110 grep -q 'Rules-Requires-Root: no' scripts/package/mkdebian 2111 history 2112 grep -q fakeroot scripts/Makefile.package scripts/package/Makefile 2113 make -j12 bindeb-pkg 2114 l 2134 cd .. 2135 l 2136 sudo dpkg -i linux-image-5.5.10_5.5.10-1_amd64.deb ... ... Kernel panic - not syncing: stack-protector : Kernel stack is corrupted in: start_secondary+0x1aa/0x1b0 # kernel panic caused by compiling kernels 5.5->5.6 with gcc 10 or higher. Only fix is to set ./linux/.config to CONFIG_STACKPROTECTOR_STRONG=n ... but 5.5.10 still works and boots, so next try 5.5.19 commit: 466fa16da5898d23d65fe6ba3c1765c39b663e9d git checkout 466fa16da5898d23d65fe6ba3c1765c39b663e9d scripts/config --enable LOCALVERSION_AUTO scripts/config --disable DEBUG_INFO scripts/config --disable SYSTEM_TRUSTED_KEYRING scripts/config --set-str SYSTEM_TRUSTED_KEYS '' make olddefconfig make -j12 bindeb-pkg # and 5.5.19 boots just fine. So it's on to ... 5.6.1 aafdf65a87bb65bc4c228f8016329de2ae95fafc # 2020-04-01 10:58:19 +0200 git checkout aafdf65a87bb65bc4c228f8016329de2ae95fafc scripts/config --enable LOCALVERSION_AUTO scripts/config --disable DEBUG_INFO scripts/config --disable SYSTEM_TRUSTED_KEYRING scripts/config --set-str SYSTEM_TRUSTED_KEYS '' make olddefconfig make -j10 bindeb-pkg ... make[2]: *** [debian/rules:6: build] Error 2 dpkg-buildpackage: error: debian/rules build subprocess returned exit status 2 make[1]: *** [scripts/Makefile.package:83: bindeb-pkg] Error 2 make: *** [Makefile:1417: bindeb-pkg] Error 2 Solution is to clear the debian files from the last build, rm -f debian/files make -j10 bindeb-pkg ... EXPORTS arch/x86/lib/lib-ksyms.o AR arch/x86/lib/built-in.a make[4]: *** [scripts/Makefile.build:505: arch/x86/kvm] Error 2 make[3]: *** [Makefile:1683: arch/x86] Error 2 make[3]: *** Waiting for unfinished jobs.... make[2]: *** [debian/rules:6: build] Error 2 dpkg-buildpackage: error: debian/rules build subprocess returned exit status 2 make[1]: *** [scripts/Makefile.package:83: bindeb-pkg] Error 2 make: *** [Makefile:1417: bindeb-pkg] Error 2 Nope, not a solution. It is a kernel bug: https://bugzilla.kernel.org/show_bug.cgi?id=207173 Edit virt/kvm/kvm_main.c about line ~2220. /* superkuh edit */ /*gfn_t nr_pages_avail;*/ gfn_t nr_pages_avail = 0; make -j10 bindeb-pkg ... and it builds! ... and 5.6.0 boots fine? # If 5.6.1 doesn't boot, next is...? I don't know. It might be time to try going back to git bisect starting from v5.5.1 instead of v5.5.0. # 5.6.2 https://git.kernel.org/pub/scm/linux/kernel/git/stable/linux.git/commit/?h=v5.6.2&id=9fbe5c87eaa9b72db08425c52c373eb5f6537a0a # 9fbe5c87eaa9b72db08425c52c373eb5f6537a0a 2020-04-02 08:02:32 +0200 git checkout 9fbe5c87eaa9b72db08425c52c373eb5f6537a0a -- Edit ./linux/.config to CONFIG_STACKPROTECTOR_STRONG=n scripts/config --enable LOCALVERSION_AUTO scripts/config --disable DEBUG_INFO scripts/config --disable SYSTEM_TRUSTED_KEYRING scripts/config --set-str SYSTEM_TRUSTED_KEYS '' -- Edit virt/kvm/kvm_main.c about line ~2220. /*gfn_t nr_pages_avail;*/ gfn_t nr_pages_avail = 0; make olddefconfig make -j10 bindeb-pkg _____________________ superkuh: btw, if you don't have the git stuff enabled in your shell prompt, I suggest adding that, it should make things clearer during bisect. something like this in your bash config: https://paste.debian.net/hidden/78a43c7b/ Just now I booted off a debian bullseye 5.10.0-2-amd in the form of gparted live USB and it worked fine. Maybe because of USB... It could be that it just has a newer 5.10.x kernel than debian, so this means whatever fix was introduced will make it to bullseye once the kernel is updated but I don't know where to look up the version actually being used on the web Ah. It might be worth compiling $latest kernel and giving that a spin since this bisect process is involving so many anyway. Yeah, that didn't work. 4.19->5.5.19 work. All above 5.6.7 fail. superkuh: if GRUB is fine that means it has access to a file-system where it reads both vmlinuz and initrd.img -- is it possible it's a problem with GRUB's placement of the initrd image in memory? I'd be very interested in helping resolve that issue (early boot is one of my specialisations) superkuh: also, possibly related, what kind of compression is being used for the initrd.img - recently distros switched from gzip to lzma and I know Debian and Ubuntu had a few issues as a result I'll select a kernel to boot, it says loading vmlinuz, then it shows "_" on a black screen forever. On pressing enter it says it can't find boot media. http://superkuh.com/bisect.txt superkuh: aha, so that could be related to the boot block that GRUB puts in memory to inform the kernel where things are I can boot 5.10 if it is from a usb stick. (ie gparted that uses debian bullseye) I'm probably just going to wipe everything and start over with EFI now that I've finally acquired a 2017 era video card. superkuh: live ISO image on USB? Yes. On USB with bios boot (not uefi). superkuh: and, is the /boot/ file-system the same as the OS rootfs or separate? OS filesystem. superkuh: thanks. I'm wondering if you would be able to create an image archive of sectors 0-33 of the boot disk, and a tar of the /boot/ directory only. I can very easily reproduce your scenario from those and experiement with it superkuh: sectors 0-33 will contain the BIOS boot-strap code and the GPT primary from which I can recreate the same partition layout (doesn't need a rootfs to replicate your issue ) TJ-, I'm willing but my initial searches of how to do that are not coming up with anything obvious. superkuh: I can tell you :) "export DEV=/dev/sdX; sudo dd if=$DEV of=/tmp/gpt.raw bs=512 count=34; tar -czf /tmp/boot.tar.gz /boot/ /tmp/gpt.raw" - I /think/ that's correct. set DEV as appropriate :) Thanks. I'll give it a try. superkuh: check contentd of resulting TAR with "tar -tvf /tmp/boot.tar.gz" to ensure the gpt.raw is in there ... http://erewhon.superkuh.com/boot.tar.gz ... superkuh: thanks. I already noticed p1 starts at sector 4096 so there is lots of free space from sector 34 to sector 4095 to insert GRUB's block list superkuh: is it possible that is where GRUB has its block-list? If so, I'd need to grab those sectors too. Ah. I did hear about that from the #grub people. When I was cloning this /dev/sda to /dev/nvme0n1 I had to --force on grub-install because it wouldn't do blocklists without it. So the original almost certainly does use block-list. superkuh: I'll see if the bootstrap sector has an offset to it ... superkuh: unless I'm misreading the GRUB source for boot.S the hexdump of sector 0 at offset 0x5C contains the low 32-bits of an LBA sector offset. At that location is 0x00cb42a0 (13320864 decimal). As partition 1 starts at sector 4096 and ends at sector 170784767 that means somehow the GRUB blocklist has been embedded in the midst of that ext4 file-system, so I'm not sure how the ^&%! it is managing to boot superkuh> ... Hm! Well, that's another reason to maybe start anew then. superkuh: I can suggest an easy to applu fix though superkuh: you've got free space from sector 34 to 4095. You can create a BIOS Boot partition there. You'd need to use gdisk's e(x)pert mode to change the sector alignment using the (l) option to 1. Then press (m) to return to the main menu. (p)rint the table and you should see "First usable sector is 34, last usable sector is...". If so, you can now create a (n) partition. It'll be number 4, accept that. Starting sector 34. For end just press Enter and it'll set it to 4095. Set type to ef02 (BIOS Boot). (p)rint again to be sure it's there at the end of the table, then (w)rite and quit. TJ-, Awesome. I'll give that a try. superkuh: At that point, from a chroot mount of /dev/sda1 at, say, /target/, if you have proc, dev, sys mounted inside /target/ appropriately, you can do "sudo chroot /target grub-install --verbose /dev/sda" and you should see and get GRUB correctly installed into sda4 and be bootable superkuh: mostly that's a function of the partition tool orunding up to put it on sector boundaries that are friendly to the arrangement of SSDs superkuh: For larger devices it is often 4096. On the NVME on my laptop it is 2048. It rarely starts at 34 unless forced, or done manually like this ___bootinfoscript____ Boot Info Script 0.78 [09 October 2019] ============================= Boot Info Summary: =============================== => Grub2 (v2.00) is installed in the MBR of /dev/sda and looks at sector 13320864 of the same hard drive for core.img. core.img is at this location and looks for (,gpt1)/boot/grub. It also embeds following components: modules --------------------------------------------------------------------------- fshelp ext2 part_gpt biosdisk --------------------------------------------------------------------------- sda1: __________________________________________________________________________ File system: ext4 Boot sector type: - Boot sector info: Operating System: Debian GNU/Linux bullseye/sid Boot files: /boot/grub/grub.cfg /etc/fstab /boot/grub/i386-pc/core.img sda2: __________________________________________________________________________ File system: ext4 Boot sector type: - Boot sector info: Operating System: Boot files: sda3: __________________________________________________________________________ File system: swap Boot sector type: - Boot sector info: ============================ Drive/Partition Info: ============================= Drive: sda _____________________________________________________________________ Disk /dev/sda: 223.57 GiB, 240057409536 bytes, 468862128 sectors Disk model: SanDisk SDSSDX24 Units: sectors of 1 * 512 = 512 bytes Sector size (logical/physical): 512 bytes / 512 bytes I/O size (minimum/optimal): 512 bytes / 512 bytes Partition Boot Start Sector End Sector # of Sectors Id System /dev/sda1 1 468,862,127 468,862,127 ee GPT GUID Partition Table detected. Partition Attrs Start Sector End Sector # of Sectors System /dev/sda1 4,096 170,784,767 170,780,672 Data partition (Linux) /dev/sda2 170,784,768 433,455,103 262,670,336 Data partition (Linux) /dev/sda3 433,455,104 468,862,094 35,406,991 Swap partition (Linux) Attributes: R=Required, N=No Block IO, B=Legacy BIOS Bootable, +=More bits set "blkid" output: ________________________________________________________________ Device UUID TYPE LABEL /dev/nvme0n1p1 4028b3fd-086a-437d-a748-bc7e222650f9 ext4 /dev/nvme0n1p2 22ef4bca-e7b8-44b5-ba4b-345fd423d3b6 ext4 /dev/nvme0n1p3 fcef60cd-bde9-4491-905f-967c9800a845 swap /dev/sda1 5a6d0cd6-6f5a-40b7-aa09-4f000e737582 ext4 /dev/sda2 e41b8d9e-f199-468a-9166-d74971ec7302 ext4 /dev/sda3 17cec9da-2d0a-43c7-a80e-1eba34f02ed6 swap ========================= "ls -l /dev/disk/by-id" output: ====================== total 0 lrwxrwxrwx 1 root root 9 May 15 16:38 ata-DRW-24F1ST_d_KMMK5G90922 -> ../../sr0 lrwxrwxrwx 1 root root 9 May 15 16:38 ata-SanDisk_SDSSDX240GG25_130535410272 -> ../../sda lrwxrwxrwx 1 root root 10 May 15 16:38 ata-SanDisk_SDSSDX240GG25_130535410272-part1 -> ../../sda1 lrwxrwxrwx 1 root root 10 May 15 16:38 ata-SanDisk_SDSSDX240GG25_130535410272-part2 -> ../../sda2 lrwxrwxrwx 1 root root 10 May 15 16:38 ata-SanDisk_SDSSDX240GG25_130535410272-part3 -> ../../sda3 lrwxrwxrwx 1 root root 13 May 15 16:38 nvme-eui.0025385c019e153a -> ../../nvme0n1 lrwxrwxrwx 1 root root 15 May 15 16:38 nvme-eui.0025385c019e153a-part1 -> ../../nvme0n1p1 lrwxrwxrwx 1 root root 15 May 15 16:38 nvme-eui.0025385c019e153a-part2 -> ../../nvme0n1p2 lrwxrwxrwx 1 root root 15 May 15 16:38 nvme-eui.0025385c019e153a-part3 -> ../../nvme0n1p3 lrwxrwxrwx 1 root root 13 May 15 16:38 nvme-Samsung_SSD_970_EVO_Plus_500GB_S58SNMFNC03957V -> ../../nvme0n1 lrwxrwxrwx 1 root root 15 May 15 16:38 nvme-Samsung_SSD_970_EVO_Plus_500GB_S58SNMFNC03957V-part1 -> ../../nvme0n1p1 lrwxrwxrwx 1 root root 15 May 15 16:38 nvme-Samsung_SSD_970_EVO_Plus_500GB_S58SNMFNC03957V-part2 -> ../../nvme0n1p2 lrwxrwxrwx 1 root root 15 May 15 16:38 nvme-Samsung_SSD_970_EVO_Plus_500GB_S58SNMFNC03957V-part3 -> ../../nvme0n1p3 lrwxrwxrwx 1 root root 9 May 15 16:38 wwn-0x5001480000000000 -> ../../sr0 lrwxrwxrwx 1 root root 9 May 15 16:38 wwn-0x5001b44970194660 -> ../../sda lrwxrwxrwx 1 root root 10 May 15 16:38 wwn-0x5001b44970194660-part1 -> ../../sda1 lrwxrwxrwx 1 root root 10 May 15 16:38 wwn-0x5001b44970194660-part2 -> ../../sda2 lrwxrwxrwx 1 root root 10 May 15 16:38 wwn-0x5001b44970194660-part3 -> ../../sda3 ================================ Mount points: ================================= Device Mount_Point Type Options /dev/nvme0n1p2 /home ext4 (rw,noatime,discard) /dev/sda1 / ext4 (rw,noatime,discard) =========================== sda1/boot/grub/grub.cfg: =========================== -------------------------------------------------------------------------------- # # DO NOT EDIT THIS FILE # # It is automatically generated by grub-mkconfig using templates # from /etc/grub.d and settings from /etc/default/grub # ### BEGIN /etc/grub.d/00_header ### if [ -s $prefix/grubenv ]; then set have_grubenv=true load_env fi if [ "${next_entry}" ] ; then set default="${next_entry}" set next_entry= save_env next_entry set boot_once=true else set default="0" fi if [ x"${feature_menuentry_id}" = xy ]; then menuentry_id_option="--id" else menuentry_id_option="" fi export menuentry_id_option if [ "${prev_saved_entry}" ]; then set saved_entry="${prev_saved_entry}" save_env saved_entry set prev_saved_entry= save_env prev_saved_entry set boot_once=true fi function savedefault { if [ -z "${boot_once}" ]; then saved_entry="${chosen}" save_env saved_entry fi } function load_video { if [ x$feature_all_video_module = xy ]; then insmod all_video else insmod efi_gop insmod efi_uga insmod ieee1275_fb insmod vbe insmod vga insmod video_bochs insmod video_cirrus fi } if [ x$feature_default_font_path = xy ] ; then font=unicode else insmod part_gpt insmod ext2 set root='hd0,gpt1' if [ x$feature_platform_search_hint = xy ]; then search --no-floppy --fs-uuid --set=root --hint-bios=hd0,gpt1 --hint-efi=hd0,gpt1 --hint-baremetal=ahci0,gpt1 5a6d0cd6-6f5a-40b7-aa09-4f000e737582 else search --no-floppy --fs-uuid --set=root 5a6d0cd6-6f5a-40b7-aa09-4f000e737582 fi font="/usr/share/grub/unicode.pf2" fi if loadfont $font ; then set gfxmode=auto load_video insmod gfxterm set locale_dir=$prefix/locale set lang=en_US insmod gettext fi terminal_output gfxterm if [ "${recordfail}" = 1 ] ; then set timeout=30 else if [ x$feature_timeout_style = xy ] ; then set timeout_style=menu set timeout=5 # Fallback normal timeout code in case the timeout_style feature is # unavailable. else set timeout=5 fi fi ### END /etc/grub.d/00_header ### ### BEGIN /etc/grub.d/05_debian_theme ### insmod part_gpt insmod ext2 set root='hd0,gpt1' if [ x$feature_platform_search_hint = xy ]; then search --no-floppy --fs-uuid --set=root --hint-bios=hd0,gpt1 --hint-efi=hd0,gpt1 --hint-baremetal=ahci0,gpt1 5a6d0cd6-6f5a-40b7-aa09-4f000e737582 else search --no-floppy --fs-uuid --set=root 5a6d0cd6-6f5a-40b7-aa09-4f000e737582 fi insmod png if background_image /usr/share/desktop-base/homeworld-theme/grub/grub-4x3.png; then set color_normal=white/black set color_highlight=black/white else set menu_color_normal=cyan/blue set menu_color_highlight=white/blue fi ### END /etc/grub.d/05_debian_theme ### ### BEGIN /etc/grub.d/10_linux ### function gfxmode { set gfxpayload="${1}" } set linux_gfx_mode= export linux_gfx_mode menuentry 'Debian GNU/Linux' --class debian --class gnu-linux --class gnu --class os $menuentry_id_option 'gnulinux-simple-5a6d0cd6-6f5a-40b7-aa09-4f000e737582' { load_video insmod gzio if [ x$grub_platform = xxen ]; then insmod xzio; insmod lzopio; fi insmod part_gpt insmod ext2 set root='hd0,gpt1' if [ x$feature_platform_search_hint = xy ]; then search --no-floppy --fs-uuid --set=root --hint-bios=hd0,gpt1 --hint-efi=hd0,gpt1 --hint-baremetal=ahci0,gpt1 5a6d0cd6-6f5a-40b7-aa09-4f000e737582 else search --no-floppy --fs-uuid --set=root 5a6d0cd6-6f5a-40b7-aa09-4f000e737582 fi echo 'Loading Linux 5.10.0-6-amd64 ...' linux /boot/vmlinuz-5.10.0-6-amd64 root=UUID=5a6d0cd6-6f5a-40b7-aa09-4f000e737582 ro quiet splash resume=UUID=17cec9da-2d0a-43c7-a80e-1eba34f02ed6 echo 'Loading initial ramdisk ...' initrd /boot/initrd.img-5.10.0-6-amd64 } submenu 'Advanced options for Debian GNU/Linux' $menuentry_id_option 'gnulinux-advanced-5a6d0cd6-6f5a-40b7-aa09-4f000e737582' { menuentry 'Debian GNU/Linux, with Linux 5.10.0-6-amd64' --class debian --class gnu-linux --class gnu --class os $menuentry_id_option 'gnulinux-5.10.0-6-amd64-advanced-5a6d0cd6-6f5a-40b7-aa09-4f000e737582' { load_video insmod gzio if [ x$grub_platform = xxen ]; then insmod xzio; insmod lzopio; fi insmod part_gpt insmod ext2 set root='hd0,gpt1' if [ x$feature_platform_search_hint = xy ]; then search --no-floppy --fs-uuid --set=root --hint-bios=hd0,gpt1 --hint-efi=hd0,gpt1 --hint-baremetal=ahci0,gpt1 5a6d0cd6-6f5a-40b7-aa09-4f000e737582 else search --no-floppy --fs-uuid --set=root 5a6d0cd6-6f5a-40b7-aa09-4f000e737582 fi echo 'Loading Linux 5.10.0-6-amd64 ...' linux /boot/vmlinuz-5.10.0-6-amd64 root=UUID=5a6d0cd6-6f5a-40b7-aa09-4f000e737582 ro quiet splash resume=UUID=17cec9da-2d0a-43c7-a80e-1eba34f02ed6 echo 'Loading initial ramdisk ...' initrd /boot/initrd.img-5.10.0-6-amd64 } menuentry 'Debian GNU/Linux, with Linux 5.10.0-6-amd64 (recovery mode)' --class debian --class gnu-linux --class gnu --class os $menuentry_id_option 'gnulinux-5.10.0-6-amd64-recovery-5a6d0cd6-6f5a-40b7-aa09-4f000e737582' { load_video insmod gzio if [ x$grub_platform = xxen ]; then insmod xzio; insmod lzopio; fi insmod part_gpt insmod ext2 set root='hd0,gpt1' if [ x$feature_platform_search_hint = xy ]; then search --no-floppy --fs-uuid --set=root --hint-bios=hd0,gpt1 --hint-efi=hd0,gpt1 --hint-baremetal=ahci0,gpt1 5a6d0cd6-6f5a-40b7-aa09-4f000e737582 else search --no-floppy --fs-uuid --set=root 5a6d0cd6-6f5a-40b7-aa09-4f000e737582 fi echo 'Loading Linux 5.10.0-6-amd64 ...' linux /boot/vmlinuz-5.10.0-6-amd64 root=UUID=5a6d0cd6-6f5a-40b7-aa09-4f000e737582 ro single echo 'Loading initial ramdisk ...' initrd /boot/initrd.img-5.10.0-6-amd64 } menuentry 'Debian GNU/Linux, with Linux 5.6.0-2-amd64' --class debian --class gnu-linux --class gnu --class os $menuentry_id_option 'gnulinux-5.6.0-2-amd64-advanced-5a6d0cd6-6f5a-40b7-aa09-4f000e737582' { load_video insmod gzio if [ x$grub_platform = xxen ]; then insmod xzio; insmod lzopio; fi insmod part_gpt insmod ext2 set root='hd0,gpt1' if [ x$feature_platform_search_hint = xy ]; then search --no-floppy --fs-uuid --set=root --hint-bios=hd0,gpt1 --hint-efi=hd0,gpt1 --hint-baremetal=ahci0,gpt1 5a6d0cd6-6f5a-40b7-aa09-4f000e737582 else search --no-floppy --fs-uuid --set=root 5a6d0cd6-6f5a-40b7-aa09-4f000e737582 fi echo 'Loading Linux 5.6.0-2-amd64 ...' linux /boot/vmlinuz-5.6.0-2-amd64 root=UUID=5a6d0cd6-6f5a-40b7-aa09-4f000e737582 ro quiet splash resume=UUID=17cec9da-2d0a-43c7-a80e-1eba34f02ed6 echo 'Loading initial ramdisk ...' initrd /boot/initrd.img-5.6.0-2-amd64 } menuentry 'Debian GNU/Linux, with Linux 5.6.0-2-amd64 (recovery mode)' --class debian --class gnu-linux --class gnu --class os $menuentry_id_option 'gnulinux-5.6.0-2-amd64-recovery-5a6d0cd6-6f5a-40b7-aa09-4f000e737582' { load_video insmod gzio if [ x$grub_platform = xxen ]; then insmod xzio; insmod lzopio; fi insmod part_gpt insmod ext2 set root='hd0,gpt1' if [ x$feature_platform_search_hint = xy ]; then search --no-floppy --fs-uuid --set=root --hint-bios=hd0,gpt1 --hint-efi=hd0,gpt1 --hint-baremetal=ahci0,gpt1 5a6d0cd6-6f5a-40b7-aa09-4f000e737582 else search --no-floppy --fs-uuid --set=root 5a6d0cd6-6f5a-40b7-aa09-4f000e737582 fi echo 'Loading Linux 5.6.0-2-amd64 ...' linux /boot/vmlinuz-5.6.0-2-amd64 root=UUID=5a6d0cd6-6f5a-40b7-aa09-4f000e737582 ro single echo 'Loading initial ramdisk ...' initrd /boot/initrd.img-5.6.0-2-amd64 } menuentry 'Debian GNU/Linux, with Linux 5.5.19' --class debian --class gnu-linux --class gnu --class os $menuentry_id_option 'gnulinux-5.5.19-advanced-5a6d0cd6-6f5a-40b7-aa09-4f000e737582' { load_video insmod gzio if [ x$grub_platform = xxen ]; then insmod xzio; insmod lzopio; fi insmod part_gpt insmod ext2 set root='hd0,gpt1' if [ x$feature_platform_search_hint = xy ]; then search --no-floppy --fs-uuid --set=root --hint-bios=hd0,gpt1 --hint-efi=hd0,gpt1 --hint-baremetal=ahci0,gpt1 5a6d0cd6-6f5a-40b7-aa09-4f000e737582 else search --no-floppy --fs-uuid --set=root 5a6d0cd6-6f5a-40b7-aa09-4f000e737582 fi echo 'Loading Linux 5.5.19 ...' linux /boot/vmlinuz-5.5.19 root=UUID=5a6d0cd6-6f5a-40b7-aa09-4f000e737582 ro quiet splash resume=UUID=17cec9da-2d0a-43c7-a80e-1eba34f02ed6 echo 'Loading initial ramdisk ...' initrd /boot/initrd.img-5.5.19 } menuentry 'Debian GNU/Linux, with Linux 5.5.19 (recovery mode)' --class debian --class gnu-linux --class gnu --class os $menuentry_id_option 'gnulinux-5.5.19-recovery-5a6d0cd6-6f5a-40b7-aa09-4f000e737582' { load_video insmod gzio if [ x$grub_platform = xxen ]; then insmod xzio; insmod lzopio; fi insmod part_gpt insmod ext2 set root='hd0,gpt1' if [ x$feature_platform_search_hint = xy ]; then search --no-floppy --fs-uuid --set=root --hint-bios=hd0,gpt1 --hint-efi=hd0,gpt1 --hint-baremetal=ahci0,gpt1 5a6d0cd6-6f5a-40b7-aa09-4f000e737582 else search --no-floppy --fs-uuid --set=root 5a6d0cd6-6f5a-40b7-aa09-4f000e737582 fi echo 'Loading Linux 5.5.19 ...' linux /boot/vmlinuz-5.5.19 root=UUID=5a6d0cd6-6f5a-40b7-aa09-4f000e737582 ro single echo 'Loading initial ramdisk ...' initrd /boot/initrd.img-5.5.19 } menuentry 'Debian GNU/Linux, with Linux 5.5.10' --class debian --class gnu-linux --class gnu --class os $menuentry_id_option 'gnulinux-5.5.10-advanced-5a6d0cd6-6f5a-40b7-aa09-4f000e737582' { load_video insmod gzio if [ x$grub_platform = xxen ]; then insmod xzio; insmod lzopio; fi insmod part_gpt insmod ext2 set root='hd0,gpt1' if [ x$feature_platform_search_hint = xy ]; then search --no-floppy --fs-uuid --set=root --hint-bios=hd0,gpt1 --hint-efi=hd0,gpt1 --hint-baremetal=ahci0,gpt1 5a6d0cd6-6f5a-40b7-aa09-4f000e737582 else search --no-floppy --fs-uuid --set=root 5a6d0cd6-6f5a-40b7-aa09-4f000e737582 fi echo 'Loading Linux 5.5.10 ...' linux /boot/vmlinuz-5.5.10 root=UUID=5a6d0cd6-6f5a-40b7-aa09-4f000e737582 ro quiet splash resume=UUID=17cec9da-2d0a-43c7-a80e-1eba34f02ed6 echo 'Loading initial ramdisk ...' initrd /boot/initrd.img-5.5.10 } menuentry 'Debian GNU/Linux, with Linux 5.5.10 (recovery mode)' --class debian --class gnu-linux --class gnu --class os $menuentry_id_option 'gnulinux-5.5.10-recovery-5a6d0cd6-6f5a-40b7-aa09-4f000e737582' { load_video insmod gzio if [ x$grub_platform = xxen ]; then insmod xzio; insmod lzopio; fi insmod part_gpt insmod ext2 set root='hd0,gpt1' if [ x$feature_platform_search_hint = xy ]; then search --no-floppy --fs-uuid --set=root --hint-bios=hd0,gpt1 --hint-efi=hd0,gpt1 --hint-baremetal=ahci0,gpt1 5a6d0cd6-6f5a-40b7-aa09-4f000e737582 else search --no-floppy --fs-uuid --set=root 5a6d0cd6-6f5a-40b7-aa09-4f000e737582 fi echo 'Loading Linux 5.5.10 ...' linux /boot/vmlinuz-5.5.10 root=UUID=5a6d0cd6-6f5a-40b7-aa09-4f000e737582 ro single echo 'Loading initial ramdisk ...' initrd /boot/initrd.img-5.5.10 } menuentry 'Debian GNU/Linux, with Linux 5.5.0-1-amd64' --class debian --class gnu-linux --class gnu --class os $menuentry_id_option 'gnulinux-5.5.0-1-amd64-advanced-5a6d0cd6-6f5a-40b7-aa09-4f000e737582' { load_video insmod gzio if [ x$grub_platform = xxen ]; then insmod xzio; insmod lzopio; fi insmod part_gpt insmod ext2 set root='hd0,gpt1' if [ x$feature_platform_search_hint = xy ]; then search --no-floppy --fs-uuid --set=root --hint-bios=hd0,gpt1 --hint-efi=hd0,gpt1 --hint-baremetal=ahci0,gpt1 5a6d0cd6-6f5a-40b7-aa09-4f000e737582 else search --no-floppy --fs-uuid --set=root 5a6d0cd6-6f5a-40b7-aa09-4f000e737582 fi echo 'Loading Linux 5.5.0-1-amd64 ...' linux /boot/vmlinuz-5.5.0-1-amd64 root=UUID=5a6d0cd6-6f5a-40b7-aa09-4f000e737582 ro quiet splash resume=UUID=17cec9da-2d0a-43c7-a80e-1eba34f02ed6 echo 'Loading initial ramdisk ...' initrd /boot/initrd.img-5.5.0-1-amd64 } menuentry 'Debian GNU/Linux, with Linux 5.5.0-1-amd64 (recovery mode)' --class debian --class gnu-linux --class gnu --class os $menuentry_id_option 'gnulinux-5.5.0-1-amd64-recovery-5a6d0cd6-6f5a-40b7-aa09-4f000e737582' { load_video insmod gzio if [ x$grub_platform = xxen ]; then insmod xzio; insmod lzopio; fi insmod part_gpt insmod ext2 set root='hd0,gpt1' if [ x$feature_platform_search_hint = xy ]; then search --no-floppy --fs-uuid --set=root --hint-bios=hd0,gpt1 --hint-efi=hd0,gpt1 --hint-baremetal=ahci0,gpt1 5a6d0cd6-6f5a-40b7-aa09-4f000e737582 else search --no-floppy --fs-uuid --set=root 5a6d0cd6-6f5a-40b7-aa09-4f000e737582 fi echo 'Loading Linux 5.5.0-1-amd64 ...' linux /boot/vmlinuz-5.5.0-1-amd64 root=UUID=5a6d0cd6-6f5a-40b7-aa09-4f000e737582 ro single echo 'Loading initial ramdisk ...' initrd /boot/initrd.img-5.5.0-1-amd64 } menuentry 'Debian GNU/Linux, with Linux 4.19.0-14-amd64' --class debian --class gnu-linux --class gnu --class os $menuentry_id_option 'gnulinux-4.19.0-14-amd64-advanced-5a6d0cd6-6f5a-40b7-aa09-4f000e737582' { load_video insmod gzio if [ x$grub_platform = xxen ]; then insmod xzio; insmod lzopio; fi insmod part_gpt insmod ext2 set root='hd0,gpt1' if [ x$feature_platform_search_hint = xy ]; then search --no-floppy --fs-uuid --set=root --hint-bios=hd0,gpt1 --hint-efi=hd0,gpt1 --hint-baremetal=ahci0,gpt1 5a6d0cd6-6f5a-40b7-aa09-4f000e737582 else search --no-floppy --fs-uuid --set=root 5a6d0cd6-6f5a-40b7-aa09-4f000e737582 fi echo 'Loading Linux 4.19.0-14-amd64 ...' linux /boot/vmlinuz-4.19.0-14-amd64 root=UUID=5a6d0cd6-6f5a-40b7-aa09-4f000e737582 ro quiet splash resume=UUID=17cec9da-2d0a-43c7-a80e-1eba34f02ed6 echo 'Loading initial ramdisk ...' initrd /boot/initrd.img-4.19.0-14-amd64 } menuentry 'Debian GNU/Linux, with Linux 4.19.0-14-amd64 (recovery mode)' --class debian --class gnu-linux --class gnu --class os $menuentry_id_option 'gnulinux-4.19.0-14-amd64-recovery-5a6d0cd6-6f5a-40b7-aa09-4f000e737582' { load_video insmod gzio if [ x$grub_platform = xxen ]; then insmod xzio; insmod lzopio; fi insmod part_gpt insmod ext2 set root='hd0,gpt1' if [ x$feature_platform_search_hint = xy ]; then search --no-floppy --fs-uuid --set=root --hint-bios=hd0,gpt1 --hint-efi=hd0,gpt1 --hint-baremetal=ahci0,gpt1 5a6d0cd6-6f5a-40b7-aa09-4f000e737582 else search --no-floppy --fs-uuid --set=root 5a6d0cd6-6f5a-40b7-aa09-4f000e737582 fi echo 'Loading Linux 4.19.0-14-amd64 ...' linux /boot/vmlinuz-4.19.0-14-amd64 root=UUID=5a6d0cd6-6f5a-40b7-aa09-4f000e737582 ro single echo 'Loading initial ramdisk ...' initrd /boot/initrd.img-4.19.0-14-amd64 } } ### END /etc/grub.d/10_linux ### ### BEGIN /etc/grub.d/20_linux_xen ### ### END /etc/grub.d/20_linux_xen ### ### BEGIN /etc/grub.d/30_os-prober ### menuentry 'Debian GNU/Linux 10 (buster) (on /dev/nvme0n1p1)' --class debian --class gnu-linux --class gnu --class os $menuentry_id_option 'osprober-gnulinux-simple-63f68a24-05ae-4068-9140-c49d097888c4' { insmod part_gpt insmod ext2 if [ x$feature_platform_search_hint = xy ]; then search --no-floppy --fs-uuid --set=root 63f68a24-05ae-4068-9140-c49d097888c4 else search --no-floppy --fs-uuid --set=root 63f68a24-05ae-4068-9140-c49d097888c4 fi linux /boot/vmlinuz-4.19.0-14-amd64 root=UUID=63f68a24-05ae-4068-9140-c49d097888c4 ro quiet splash resume=UUID=fcef60cd-bde9-4491-905f-967c9800a845 initrd /boot/initrd.img-4.19.0-14-amd64 } submenu 'Advanced options for Debian GNU/Linux 10 (buster) (on /dev/nvme0n1p1)' $menuentry_id_option 'osprober-gnulinux-advanced-63f68a24-05ae-4068-9140-c49d097888c4' { menuentry 'Debian GNU/Linux (on /dev/nvme0n1p1)' --class gnu-linux --class gnu --class os $menuentry_id_option 'osprober-gnulinux-/boot/vmlinuz-4.19.0-14-amd64--63f68a24-05ae-4068-9140-c49d097888c4' { insmod part_gpt insmod ext2 if [ x$feature_platform_search_hint = xy ]; then search --no-floppy --fs-uuid --set=root 63f68a24-05ae-4068-9140-c49d097888c4 else search --no-floppy --fs-uuid --set=root 63f68a24-05ae-4068-9140-c49d097888c4 fi linux /boot/vmlinuz-4.19.0-14-amd64 root=UUID=63f68a24-05ae-4068-9140-c49d097888c4 ro quiet splash resume=UUID=fcef60cd-bde9-4491-905f-967c9800a845 initrd /boot/initrd.img-4.19.0-14-amd64 } menuentry 'Debian GNU/Linux, with Linux 4.19.0-14-amd64 (on /dev/nvme0n1p1)' --class gnu-linux --class gnu --class os $menuentry_id_option 'osprober-gnulinux-/boot/vmlinuz-4.19.0-14-amd64--63f68a24-05ae-4068-9140-c49d097888c4' { insmod part_gpt insmod ext2 if [ x$feature_platform_search_hint = xy ]; then search --no-floppy --fs-uuid --set=root 63f68a24-05ae-4068-9140-c49d097888c4 else search --no-floppy --fs-uuid --set=root 63f68a24-05ae-4068-9140-c49d097888c4 fi linux /boot/vmlinuz-4.19.0-14-amd64 root=UUID=63f68a24-05ae-4068-9140-c49d097888c4 ro quiet splash resume=UUID=fcef60cd-bde9-4491-905f-967c9800a845 initrd /boot/initrd.img-4.19.0-14-amd64 } menuentry 'Debian GNU/Linux, with Linux 4.19.0-14-amd64 (recovery mode) (on /dev/nvme0n1p1)' --class gnu-linux --class gnu --class os $menuentry_id_option 'osprober-gnulinux-/boot/vmlinuz-4.19.0-14-amd64-root=UUID=63f68a24-05ae-4068-9140-c49d097888c4 ro single-63f68a24-05ae-4068-9140-c49d097888c4' { insmod part_gpt insmod ext2 if [ x$feature_platform_search_hint = xy ]; then search --no-floppy --fs-uuid --set=root 63f68a24-05ae-4068-9140-c49d097888c4 else search --no-floppy --fs-uuid --set=root 63f68a24-05ae-4068-9140-c49d097888c4 fi linux /boot/vmlinuz-4.19.0-14-amd64 root=UUID=63f68a24-05ae-4068-9140-c49d097888c4 ro single initrd /boot/initrd.img-4.19.0-14-amd64 } } ### END /etc/grub.d/30_os-prober ### ### BEGIN /etc/grub.d/30_uefi-firmware ### ### END /etc/grub.d/30_uefi-firmware ### ### BEGIN /etc/grub.d/40_custom ### # This file provides an easy way to add custom menu entries. Simply type the # menu entries you want to add after this comment. Be careful not to change # the 'exec tail' line above. ### END /etc/grub.d/40_custom ### ### BEGIN /etc/grub.d/41_custom ### if [ -f ${config_directory}/custom.cfg ]; then source ${config_directory}/custom.cfg elif [ -z "${config_directory}" -a -f $prefix/custom.cfg ]; then source $prefix/custom.cfg; fi ### END /etc/grub.d/41_custom ### -------------------------------------------------------------------------------- =============================== sda1/etc/fstab: ================================ -------------------------------------------------------------------------------- # /etc/fstab: static file system information. # # tmpfs /tmp tmpfs defaults,noatime,mode=1777 0 0 #Entry for /dev/sda1 : #UUID=5a6d0cd6-6f5a-40b7-aa09-4f000e737582 / ext4 defaults,noatime,discard 0 1 UUID=4028b3fd-086a-437d-a748-bc7e222650f9 / ext4 defaults,noatime,discard 0 1 #Entry for /dev/sda2 : #UUID=e41b8d9e-f199-468a-9166-d74971ec7302 /home ext4 defaults,noatime,discard 0 2 UUID=22ef4bca-e7b8-44b5-ba4b-345fd423d3b6 /home ext4 defaults,noatime,discard 0 2 #Entry for /dev/sda3 : #UUID=17cec9da-2d0a-43c7-a80e-1eba34f02ed6 swap swap defaults,noatime,discard 0 2 UUID=fcef60cd-bde9-4491-905f-967c9800a845 swap swap defaults,noatime,discard 0 2 -------------------------------------------------------------------------------- =================== sda1: Location of files loaded by Grub: ==================== GiB - GB File Fragment(s) 10.488292694 = 11.261718528 boot/grub/grub.cfg 3 6.351905823 = 6.820306944 boot/grub/i386-pc/core.img 1 66.135250092 = 71.012184064 boot/vmlinuz-4.19.0-14-amd64 1 21.145027161 = 22.704300032 boot/vmlinuz-5.10.0-6-amd64 1 14.284648895 = 15.338024960 boot/vmlinuz-5.5.0-1-amd64 1 21.804134369 = 23.412011008 boot/vmlinuz-5.5.10 1 6.409606934 = 6.882263040 boot/vmlinuz-5.5.19 1 23.686996460 = 25.433718784 boot/vmlinuz-5.6.0-2-amd64 1 23.686996460 = 25.433718784 vmlinuz 1 14.284648895 = 15.338024960 vmlinuz.old 1 3.681144714 = 3.952599040 boot/initrd.img-4.19.0-14-amd64 3 57.799102783 = 62.061314048 boot/initrd.img-5.10.0-6-amd64 1 14.805049896 = 15.896801280 boot/initrd.img-5.5.0-1-amd64 1 22.703075409 = 24.377241600 boot/initrd.img-5.5.10 1 7.007762909 = 7.524528128 boot/initrd.img-5.5.19 1 24.469417572 = 26.273837056 boot/initrd.img-5.6.0-2-amd64 1 24.469417572 = 26.273837056 initrd.img 1 14.805049896 = 15.896801280 initrd.img.old 1 ____MBR on GPT tutorial____ When GRUB boots from a MBR, the number of legacy BIOS compatibility steps it needs to take at the beginning of the boot process means that the code actually in the MBR is only capable of loading one disk block whose LBA number is patched in to the MBR code at the time of installation. That block is usually the first block of GRUB core image. It contains the code to load more blocks, and a list of block numbers that defines where the rest of the GRUB core image is located. On a MBR-partitioned disk, there is usually unused space between the MBR and the start of the first partition. With MS-DOS, the original convention was to start the first partition at the start of the next disk track, which usually means there will be at least 63 disk blocks before the first partition, including the MBR. On modern systems, the first MBR partition is now more commonly placed to start at exactly 1MiB from the beginning of the disk, i.e. at block #2048, to optimize the data alignment for disks, SSDs and SAN storage systems that may internally use a block size bigger than 512 bytes. So, on a MBR-partitioned disk, the beginning of the disk is normally arranged like this: block #0: MBR block #1: first block of GRUB core image, contains the block list blocks #2...#n: the rest of the GRUB core image block #2048: beginning of the first partition. Note that the loading of the GRUB core image works exclusively by pre-determined block numbers: until the GRUB core image is fully loaded and extracted, GRUB will have no awareness of partition tables nor filesystems of any kind. On a GPT-partitioned disk, the blocks immediately after block #0 are occupied by the GPT partition table, so the GRUB core image is embedded into a "BIOS boot partition" instead. That just means the block number embedded into the MBR will not be 1, but instead the number of the first block of the BIOS boot partition, and the rest of the blocks belonging to the core image will be likewise shifted. So on a GPT-partitioned disk with BIOS-style GRUB on it, the physical layout will be something like this, assuming the BIOS boot partition is the first one on the disk: block #0: GPT protective MBR, with GRUB MBR code embedded blocks #1...#(x-1): actual GPT partition table block #x: the first block of the BIOS boot partition, contains first block of GRUB core image with the block list blocks #(x+1)...#(x+n): the rest of the GRUB core image The fact that you can still get to GRUB rescue mode indicates that although you said you removed your BIOS boot partition, you did not yet overwrite its blocks; although the space occupied by the BIOS boot partition may now be unallocated space between partitions, or unused space in another resized partition, it still has its old contents, and GRUB can still load those blocks and find its core image. But nothing in particular shields those blocks now from getting overwritten: as soon as that happens for whatever reason, the GRUB core image will be destroyed and GRUB will fail to get even as far as the rescue mode. Contents of the GRUB core image The GRUB core image contains several things: the GRUB kernel: this is the only part technically required to get into GRUB rescue mode. embedded initial GRUB root path, to indicate which disk, partition and directory within it holds the GRUB configuration file and the GRUB modules directory. In Linux, these normally appear as /boot/grub/grub.cfg and /boot/grub/i386-pc respectively when the Linux system is running normally. a set of embedded GRUB modules, containing at least the code to read and understand the partition table and the filesystem type used on the partition referred to by the initial GRUB root path. As the core image may need to fit into less than 63 disk blocks, this set of modules is usually kept as minimal as possible on MBR systems. optionally, an embedded GRUB configuration file with one or more GRUB commands optionally, an embedded disk image, similar to the ones used by the memdisk tool of the SYSLINUX bootloader family optionally, a GPG public key used to sign other GRUB modules and the OS kernel for security (and to satisfy Secure Boot acceptance requirements on UEFI versions of GRUB) All this is LZMA compressed to minimize its size, so it cannot be easily read or modified manually.