yandex
loader

please wait

  • Jignesh Patel Feb-12-2019 06:20:19 AM ( 3 months ago )

    I transferred the output tensor to GPU at different positions in my code(before or after modifying its value) but got different results. What's the reason?

    The failed code can be simplified as:

    def Network(self):
        ........
        A = self.model(input)
        indexlist = self.indexlist
        output = torch.zeros(A.size(0))
        for i,li in enumerate(indexlist):
            if li:
                s,e = li
                output[i]+=sum(A[i,s:e])
        output = output if self.no_cuda else output.cuda(device=self.gpu,async=True)
        return output
    pred = Network()
    loss = F.nll_loss(pred,target)
    loss.backward()
    

    And the RuntimeError: Function torch::autograd::CopySlices returned an invalid gradient at index 1 - expected type torch.cuda.FloatTensor but got torch.FloatTensor

    If I changed one line as follows, it runs normally:

    def Network(self):
        ........
        A = self.model(input)
        indexlist = self.indexlist
        output = torch.zeros(A.size(0))
        output = output if self.no_cuda else output.cuda(device=self.gpu,async=True)
        for i,li in enumerate(indexlist):
            if li:
                s,e = li
                output[i]+=sum(A[i,s:e])
        return output
    pred = Network()
    loss = F.nll_loss(pred,target)
    loss.backward()

Please login

Similar Discussion

Recommended For You