The simple answer is yes, but to how much it matters depends on your organization. If you are a small group of people that all work in the same office it might not matter as much but if you organization has multiple offices and/or remote workers then that My Site could be the first thing they see about you. When I look at a person's My Site and it has no information it gives me a lukewarm feeling about this person. We have entered a time when people meet you online first and in person second, and what they learn about you online can greatly influence how that person perceives you. Your My Site could affect your career progression in ways you never even thought about. Unfortunately this can go both ways. So here are some My Site tips.
Use a professional looking head shot.
Be thorough but don't come off as bragging
Keep it work related, but include items that other workers might have similar interests in
Keep your personal life out of it
Try to enter some information in each of the fields provided
That being said, spend some time on your My Site. More people are looking at it than you know.