Marijuana arrest without possession?

I got stopped by the cops for wearing a bandanna with pot leaves on it, they searched me and found $130 and went through my cell phone, finding weed related texts, then confied both. Did they have a right to stop and search me, then tell me I'll be getting charges in the mail if I didn't have any actual marijuana on me? He accused me of smelling like weed, but I just got out of the shower 5 minutes before, and there was no possible way. I'm just trying to get my money and phone back, and not be charged with anything when they just randomly stopped me.