If anyone is interested, I've submitted a patch to Rob Brown (maintainer).
The patch is available at:
(can't link it; filename must contain characters the link parser can't handle)
Just pass the options sock_owner and/or sock_group to Net::Server to change the owner and group (respectively) of unix sockets. If either or both options are omitted, the previous behavior is unchanged.
Hope someone finds this useful.