kernel panics when hot removing U.2 nvme disk

[Meng Wang]

> -----Original Message-----
> From: Christoph Hellwig <hch at infradead.org>
> Sent: Wednesday, September 23, 2020 10:13 PM
> To: Keith Busch <kbusch at kernel.org>
> Cc: Meng Wang <meng at hcdatainc.com>; linux-ext4 at vger.kernel.org; linux-
> nvme at lists.infradead.org
> Subject: Re: kernel panics when hot removing U.2 nvme disk
> 
> On Fri, Sep 18, 2020 at 06:44:01PM -0700, Keith Busch wrote:
> > On Fri, Sep 18, 2020 at 11:47:27PM +0000, Meng Wang wrote:
> > > Hi,
> > > We found kernel panics today when doing test on hot remove U.2 nvme
> > > disk. After hot remove the nvme disk (formatted as ext4), the system
> > > freezes and all services stuck. Lot of kernel message flushed the
> > > syslog, including the CPU soft lockup, ext4 NULL point er dereferece
> > > and ib nic transmission timeout. The kernel panics and configuration
> > > are shown below. The used kernel is 5.4.0-050400-generic and OS is
> > > Ubuntu 16.04. Not sure whether it's a known bug or configuration
> > > error. Any advise are welcome.
> >
> > [cc'ing ext4 mailing list]
> >
> > The NULL dereference occured before the soft lockup, so I'm guessing the
> > Oops'ed process is holding the same lock the removal task wants.
> >
> > Your kernel is a bit older, so it may be worth verifying if your
> > observation still occurs on the current stable or current mainline, but
> > the ext4 developers may have a better idea as this doesn't at least
> > initially appear specific to nvme.
> 
> The problem is the crazy __invalidate_device stuff that calls into
> file system eviction from all kinds of super critical block paths.
> While I haven't debugged the root cause this kind of thing just causes
> problems without really helping anyone.  I have a half-finished series
> that kills this crap and instead allows the file system (or other
> block device user) to pass shutdown and resize callbacks when the
> exclusively open a block device.  That way the file system driver
> can just mark the file system shutdown to prevent any further damage
> without all this mess.

Thanks for the info. Is it a problem solely for ext4 + nvme combination? If we change file system or use SATA drive, will the problem get workaround?