Basically what it says on the tin. Having read though some of the materials on the issue, I am baffled by how recklessly the word is used, given the consequences of such usage.
Pedophiles are the people with sexual attraction to prepubescent children. It doesn't matter whether they do or don't act on that attraction; in fact, many don't. It is a sexual interest/mental condition that cannot be reliably changed.
Child molesters, on the other hand, are not necessarily pedophiles - in fact, 50 to 75% of child molesters do not have pedophilic interest.
Both facts can be sourced from the respective Wikipedia article and more info can be found in respective research.
Why does this matter?
Because the current use of the word reinforces stigma around pedophilia and makes it less likely for people with pedophilic disorder to reach out for help for the fear they would be outed and treated the same as actual child abusers.
This, in turn, makes those in a vulnerable position more likely to cross the line and get into the category of child abusers instead of coming for help. Also, it heavily affects people who did nothing to deserve such treatment.
What should we do?
We should leave the word "pedophile" to the context in which it belongs, which is the mental health and sexuality spheres, and avoid using the term to describe sexual offenders against minors. At the very least, one would most likely be wrong. At most, one would participate in the cycle of child abuse.
When I think about how many famous, powerful people have been outed as child molesters, it definitely seems to me that pedophilia is something that can develop within people. I don't think that pedophiles tend to become famous, nor do I think that the trend among powerful people is reflected in the general population.
From the wikipedia article
This seems crazy to me and I don't think any of it explains why so many powerful people actively seek to molest children. Treating pedophilia as a condition or as a quirk in one's sexuality seems dangerous to me. You can like petite people, you can do schoolgirl roleplay or whatever you want with a consenting adult. Don't even think about kids. That should be an immediate red light in anyone's brain.
Edit: nah I think OP's cooking actually, after thinking about it. If it helps pedophiles seek help then who am I to insist on calling child molesters pedophiles?
But, it is an actual mental condition/mental illness/mental disorder. Just like what op said, there are, in fact, a lot of people that are attracted to kids that know it's wrong and don't act on those thoughts. They can not control having those thoughts in the same way that I can see a beautiful woman and am immediately attracted to them.
In some peoples' brains it's just not. Attraction works in a way that doesn't ask you for your stance.
And among wealthy and powerful, there are plenty of sociopaths who would abuse a child without being pedophilic per se. Also, if you check the age of their victims, this is most commonly post-puberty minors, and attraction to those is highly common in an adult population.
There is currently no evidence that pedophilia can be developed, except for the fact that there's an elevated percentage of pedophiles among people who have survived child sexual abuse as minors themselves.