BUG_ON at nvme_setup_prp getting hit.

[Hrishikesh Gokhale]

Thanks for your reply Keith.

Warm Regards,
Hrishikesh Gokhale

On Tue, Jan 7, 2014 at 11:45 PM, Keith Busch <keith.busch at intel.com> wrote:
> On Tue, 7 Jan 2014, Hrishikesh Gokhale wrote:
>>
>> Hi All,
>>
>> Requesting for help in resolving the following:
>>
>> Setup:
>> Running FIO tester tool on NVMe device driver (/block/nvme.c) with I/O
>> engine “libaio”. Using Odd block size like 17K, 129K, 257K,  etc.
>> Distribution ‘Fedora 18’ (Kernel v3.6.10).
>>
>> Observation:
>> I’ve observed intermittent hit on BUG_ON which is pointing to
>> nvme_setup_prps() function in nvme.c driver.
>>
>>>>> CODE
>>
>> static int nvme_setup_prps(...)
>>
>> {
>>       int dma_len = sg_dma_len(sg);
>>
>>       u64 dma_addr = sg_dma_address(sg);
>>
>>       int offset = offset_in_page(dma_addr);
>>       ...
>>       dma_len -= PAGE_SIZE;
>>       ...
>>       if (dma_len > 0)
>>           continue;
>>       BUG_ON(dma_len < 0);
>> }
>> <<< CODE
>>
>> After adding prints , I found that this problem is reproduced on use
>> of PRP list.
>>
>> Offset value in DMA page is changed (in nvme_setup_prps() function)
>> from the original offset value which was present while sg_set_page()
>> function was executed. The offset value of virtual address is found
>> different than the offset value of physical/DMA address.
>
>
> This BUG_ON shouldn't happen unless we gave nvme_setup_prps an sgl that
> can't be mapped to a PRP list, so the request should have been split
> instead. A virtual address with different alignment than the dma address
> might cause that and sounds similar to this issue from way back:
>
> merlin.infradead.org/pipermail/linux-nvme/2012-November/000121.html
>
> I didn't see that anything came of it, though.
>
> Maybe our BIOVEC_NOT_VIRT_MERGEABLE check should use the physical
> addresses instead of virtual to see if the bio_vec is usable. ?
>
>
>> Query:
>> Is it possible to avoid the ‘offset value change’ in DMA-able memory
>> (in nvme_setup_prps() function) to avoid BUG_ON()check? ”
>
>
>> Thanks & Regards,
>> Hrishikesh