You've got nothing but their word that Saemon stole it from them. For all you know, he could've got it from its rightful owner and the Githyanki could just be thieves.
Considering what you know of Saemon and the person whom he says gave it to him (Bodhi), a party of nobodies would be paragons of truth in comparison. Unless, of course, you've taken the ship from Spellhold and you know what Githyanki do.
Question: Do you even get the Blade at all if you take the portal? It's been so long since I've gone that route that I forget.
Just regarding a little that topic, keeping the Blade
should incur a virtue hit, because:
a) the one who gave it to you, Saemon, is (from all you know) a untrustworthy scoundrel who probably wanted to get rid of it;
b) the Githyanki requested the blade back about 3 times, from all places - clearly it belongs to them (somehow). If they attack you, then that's because of their warlike nature;
c) keeping the blade is greed, because by that time you get lots of magic items and it comes as a free gift anyway. You know it's the right blade, thanks to the Item Description Scroll anyway;
Since they were appearing everywhere for their blade, without attacking me outright, I was sure I owed them the artifact.
I just freed Gont of Riativin from the imprisonment orb thing in the Underdark, and let him go without a fight. If there's a virtue hit for killing him, why not a virtue gain for sparing him?
I'd also like to propose that killing any innocent person would incur the same penalty, whether that be Joe Schmoe commoner, Drizzt T. Heroman, or an ungrateful jerk you meet in a big cave.
I don't know if freeing any madman should be virtuous UNLESS that madman actually suffered/was cast upon and went mad at everyone (similar to Brage from BG1).