Fancying someone is physical alone.

You can find someone attractive based on other factors, you could fancy someone based on their picture then find them unattractive in person due to their personality and behaviour.