front window being able to push in

I noticed I got some rattling on my driver side front window when it's down a few inches. I decided to inspect it and noticed that I'm able to push it in a bit with the window all the way up. The passenger side pushes in, but a lot less. Is this normal?

I pushed the panel in and it seemed to help a bit as far as the depth I'm able to push in. I'm just afraid of something like water getting in the door between the window and molding.