Is this being too impulsive (i'm pursuing a guy)?

i want to ask him if he wants to hang out over the summer. we don't know each other very well but we say hi and wave whenever we p each other in the halls and we had one cl together. we don't talk much, and i know he wanted to ask this girl who he's liked for quite a while to prom but she already had a date. should i ask him if he wants to hang out, or will that sort of freak him out? if i do and he agrees, should it just be the two of us or some other friends as well (i don't really have that many. that's why i consider him one of my closer friends)