Scooter Braun has insisted his public falling out with Taylor Swift didnt "affect" him and only his divorce had a real impact.