Wenn you've got the amulet and are on your way back, there should be a sudden meeting with Kalthar who fights against you. Wasn't he there?