Only Agoraphobia's controller can activate its last ability, no matter who controls the creature Agoraphobia's attached to.

Agoraphobia's last ability can be activated only while it's on the battlefield.

Players don't have priority to cast spells and activate abilities between combat damage being assigned and being dealt. This means that if you want to return Agoraphobia to its owner's hand before combat damage is dealt, you must do so before combat damage is assigned (and the creature will no longer get -5/-0).