Won’t harm the car, just keep in mind if it starts to roll faster and you want to hop on and hit the brakes they will be unassisted by vacuum and will be tougher.

I do it all the time when I’ve backed up too close to my wall and have to get in the boot.