Hi Bradley,
It may be rest bounce back hitting the fob on the way out and the air pulling it off once it gets downrange. Try putting a little powder or something else to check for contact, lipstick will work too. Also check the fob to see if the edges are getting beat up. The Rest Rocket might be something that will help with bounce back.
Hope this helps, Roger