nemanin

cross-posted from: https://lemmy.world/post/16636012

Me again, back with another probably dumb question, but you beautiful bastards have been so helpful so far, I can't stay away!

I got 10x 10TB SAS drives from FB market place. They look like they're in good shape and the guy says he pulled them from the live server of a family member who passed. HGST. most/all are 2018.

I brought them home and tried to mount them one-by-one in an xpenology VM to smart test them (easiest place I had set up for SMART tests).

But most of my troubleshooting has just involved looking at the HBA menus in BIOS and seeing if the drives even show up. Currently only 1 seems to reliably.

and I got a weird mix of drive showing up fine, but others not showing up at all. I also got a couple drives that passed a SMART test, then when I pulled them and tried to remount them later, they don't even show up?

I tried using molex to SATA power adapters to rule out 3.3v, didn't help.

I don't think it's formatting because some of them mounted at least once and they all came from the same server.

I tried putting the HBA in another PCIe slot, plan to try the third slot tonight.

I have this HBA, confirmed in BIOS it's in IT mode: https://www.amazon.com/gp/product/B0BYZBNXBS/

(I'm having troubles finding a good manual for this board, by the way. there are flashing LEDs that may be trying to tell me something?)

and these breakout cables: https://www.amazon.com/gp/product/B07B9SBSVW/

I might try another HBA, rule out bad board. I plan to try the third PCIe slot tonight, try to rule that out...

What else?

They could be just bad drives, but the seller seemed genuine and they look like they're in good shape. He even pinged me after the sale to see how they worked out for me.. doesn't seem like a scammer.

Also, a couple questions: 1) these should be hot-swappable, right? and 2) what would happen if this PCIe x8 card is in a PICE x4 slot?

Thanks again. You guys have been great! :)

Me again, back with another probably dumb question, but you beautiful bastards have been so helpful so far, I can't stay away!

I got 10x 10TB SAS drives from FB market place. They look like they're in good shape and the guy says he pulled them from the live server of a family member who passed. HGST. most/all are 2018.

I brought them home and tried to mount them one-by-one in an xpenology VM to smart test them (easiest place I had set up for SMART tests).

But most of my troubleshooting has just involved looking at the HBA menus in BIOS and seeing if the drives even show up. Currently only 1 seems to reliably.

and I got a weird mix of drive showing up fine, but others not showing up at all. I also got a couple drives that passed a SMART test, then when I pulled them and tried to remount them later, they don't even show up?

I tried using molex to SATA power adapters to rule out 3.3v, didn't help.

I don't think it's formatting because some of them mounted at least once and they all came from the same server.

I tried putting the HBA in another PCIe slot, plan to try the third slot tonight.

I have this HBA, confirmed in BIOS it's in IT mode: https://www.amazon.com/gp/product/B0BYZBNXBS/

(I'm having troubles finding a good manual for this board, by the way. there are flashing LEDs that may be trying to tell me something?)

and these breakout cables: https://www.amazon.com/gp/product/B07B9SBSVW/

I might try another HBA, rule out bad board. I plan to try the third PCIe slot tonight, try to rule that out...

What else?

They could be just bad drives, but the seller seemed genuine and they look like they're in good shape. He even pinged me after the sale to see how they worked out for me.. doesn't seem like a scammer.

Also, a couple questions: 1) these should be hot-swappable, right? and 2) what would happen if this PCIe x8 card is in a PICE x4 slot?

Thanks again. You guys have been great! :)

Me again. The guy with the NIC problem from before.

I installed the Rx590 and it shows up in lspci as an RTX 2070. I was hoping it was just Proxmox not having drivers or something, but when I pass it into the Hackintosh it's meant for, it shows as NVIDIA there too:

Now I did get the Rx590 off eBay, but I'm it was listed as and looks like this: https://www.powercolor.com/product?id=1551768831

So I think it is actually a Rx590.

This is lspci. 01:00.0 is a RTX 3060. 03:00.0 is the Rx590.

00:00.0 Host bridge: Intel Corporation Device a740 (rev 01)

00:01.0 PCI bridge: Intel Corporation Device a70d (rev 01)

00:02.0 VGA compatible controller: Intel Corporation Raptor Lake-S GT1 [UHD Graphics 770] (rev 04)

00:14.0 USB controller: Intel Corporation Alder Lake-S PCH USB 3.2 Gen 2x2 XHCI Controller (rev 11)

00:14.2 RAM memory: Intel Corporation Alder Lake-S PCH Shared SRAM (rev 11)

00:15.0 Serial bus controller: Intel Corporation Alder Lake-S PCH Serial IO I2C Controller #0 (rev 11)

00:16.0 Communication controller: Intel Corporation Alder Lake-S PCH HECI Controller #1 (rev 11)

00:17.0 SATA controller: Intel Corporation Alder Lake-S PCH SATA Controller [AHCI Mode] (rev 11)

00:1a.0 PCI bridge: Intel Corporation Alder Lake-S PCH PCI Express Root Port #25 (rev 11)

00:1b.0 PCI bridge: Intel Corporation Device 7ac4 (rev 11)

00:1c.0 PCI bridge: Intel Corporation Alder Lake-S PCH PCI Express Root Port #2 (rev 11)

00:1c.4 PCI bridge: Intel Corporation Alder Lake-S PCH PCI Express Root Port #5 (rev 11)

00:1d.0 PCI bridge: Intel Corporation Alder Lake-S PCH PCI Express Root Port #9 (rev 11)

00:1f.0 ISA bridge: Intel Corporation Z690 Chipset LPC/eSPI Controller (rev 11)

00:1f.3 Audio device: Intel Corporation Alder Lake-S HD Audio Controller (rev 11)

00:1f.4 SMBus: Intel Corporation Alder Lake-S PCH SMBus Controller (rev 11)

00:1f.5 Serial bus controller: Intel Corporation Alder Lake-S PCH SPI Controller (rev 11)

00:1f.6 Ethernet controller: Intel Corporation Ethernet Connection (17) I219-V (rev 11)

01:00.0 VGA compatible controller: NVIDIA Corporation GA106 [GeForce RTX 3060] (rev a1)

01:00.1 Audio device: NVIDIA Corporation GA106 High Definition Audio Controller (rev a1)

02:00.0 Non-Volatile memory controller: Sandisk Corp Western Digital WD Black SN850X NVMe SSD (rev 01)

03:00.0 VGA compatible controller: NVIDIA Corporation TU106 [GeForce RTX 2070] (rev a1)

03:00.1 Audio device: NVIDIA Corporation TU106 High Definition Audio Controller (rev a1)

03:00.2 USB controller: NVIDIA Corporation TU106 USB 3.1 Host Controller (rev a1)

03:00.3 Serial bus controller: NVIDIA Corporation TU106 USB Type-C UCSI Controller (rev a1)

04:00.0 Ethernet controller: Realtek Semiconductor Co., Ltd. RTL8125 2.5GbE Controller (rev 05)

05:00.0 Non-Volatile memory controller: Micron/Crucial Technology Device 5415 (rev 01)

06:00.0 PCI bridge: PLX Technology, Inc. PEX 8724 24-Lane, 6-Port PCI Express Gen 3 (8 GT/s) Switch, 19 x 19mm FCBGA (rev ca)

07:00.0 PCI bridge: PLX Technology, Inc. PEX 8724 24-Lane, 6-Port PCI Express Gen 3 (8 GT/s) Switch, 19 x 19mm FCBGA (rev ca)

Like the title says. I installed a GPU, everything posts and boots fine. The lights on the Ethernet port are lit up and will stay lit up indefinitely (I assume) if I leave it at the kernel select screen.

But as soon as I load a kernel, the lights go dark. It also is not shown as an active client on my gateway, so it’s not working at all.

I’ve tried lots of commands I’ve found to force it up. It looks to me like the NIC assigned to vmbr0 is correct. Etc. I just can’t get it to work.

If I remove the GPU, it immediately works again. NIC stays up after the kernel loads and I can access the web UI as normal.

rooteprox. *

root@prox:*# ip a

1. 10: «LOOPBACK, UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000
   link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
   inet 127.0.0.1/8 scope host lo
   valid_lft forever preferred_lft forever
   inet6 :: 1/128 scope host noprefixroute valid_lft forever preferred_lft forever
2. enpsso: ‹BROADCAST, MULTICAST> mtu 1500 qdisc noop state DOHN group default qlen 1000 link/ether a8:a1:59:be:f2:33 brd ff:ff:ff:ff:ff:ff
   enp0s31f6: «NO-CARRIER, BROADCAST, MULTICAST, UP> mtu 1500 qdisc pfifo_fast master vmbro state DOWN group default qlen 1000 link/ether a8:a1:59:be:f2:32 brd ff:ff:ff:ff:ff:ff
   vmbrO: ‹NO-CARRIER, BROADCAST, MULTICAST, UP> mtu 1500 qdisc noqueue state DOWN group default qlen 1000 link/ether a8:a1:59:be:f2:32 brd ff:ff:ff:ff:ff:ff
   inet 192.168.1.3/24 scope global vmbro valid_lft forever preferred_lft forever

root@prox: *# cat /etc/network/interfaces

auto lo

iface lo inet loopback

iface enp0s31f6 inet manual

auto vmbro

iface vmbro inet static

address 192.168.1.3/24

gateway 192.168.1.1

bridge-ports enp0s31f6

bridge-stp off bridge-fd o

iface enps0 inet manual

source /etc/network/interfaces.d/*

root@prox: ~# service network restart

Failed to restart network.service: Unit network.service not found.