No one replied to my posting but I figured it out: looks like the new zones weren't associated with the delegation set and thus could not be found when queried. I had to delete all zones, set the default delegation again, and sync it all over.
It seems that both the extension and the Route53 console can show that a domain is indeed synced, but cannot show where it was "delegated" to. Also it seems that the extension cannot change a delegation set for a domain, in case it ends up in the wrong set. Worse yet is that it takes quite a few minutes to delete all zones then sync them again, if you ever need to refresh everything.