Backpropagated gradients are rotated and scaled by the Jacobian at each step. The copy gradient simply reuses the reward gradient everywhere. Click to reposition, drag the dial to rotate the reward gradient.