The source of combat damage is the creature that deals it.

If a spell is causing damage to be dealt, that spell will always identify the source of the damage. In most cases, the source is the spell itself. For example, Burn Trail says “Burn Trail deals 3 damage to target creature or player.”

If an ability is causing damage to be dealt, that ability will always identify the source of the damage. The ability itself is never the source. However, the source of the ability is often the source of the damage. For example, Knollspine Invocation’s ability says “Knollspine Invocation deals X damage to target creature or player.”

If the source of the damage is a permanent, Swans of Bryn Argoll checks who that permanent’s controller is at the time that damage is prevented. If the permanent has left the battlefield by then, its last known information is used. If the source of the damage is a spell, its controller is obvious. If the source of the damage is a card from some other zone (such as a cycled Gempalm Incinerator), Swans of Bryn Argoll checks its owner rather than its controller.

If a creature with wither would deal damage to Swans of Bryn Argoll, that damage is treated just like any other damage. It’s prevented, and the creature’s controller draws cards. Swans of Bryn Argoll doesn’t get any -1/-1 counters.