I'm trying to crochet a hat, and I keep ending up with more at the end?

Where my start/finish round spot is, it's a little longer than the rest of the hat. How do I fix this? What am I doing wrong?