Most manufacturers have separate product families for "servo drives" and "variable frequency drive" or "inverter". However, if we take one of the many VFDs that support PM motors with sensor and a PM servo drive, the control, is the same up to the speed loop. Both servo and VFD (except for DTC) need an inner vector current control loop, which means torque control, and a speed control regulator. They also need a self-tuning procedure (motor identification and sensor phasing).
One should define the application since there are various different types of servo motors and some of them require complex and very specific control that is best offered by a purpose matched servo drive. There are variable frequency drives with motion capability and I expect that in the near future the VFD will be able to do numerous of the motion functions of standard servo drives.
If the VFD has firmware available to control a PM motor. Many VFDs have this option. The problem is that you are only able to control the 'speed loop', the VFD accuracy is not sufficient to be a part of the 'position loop' to the same level as a servo drive.
Are power electronics and control HW so different between the two? AFAIK, they are similar. Is position loop/trajectory planning impossible to implement on the same control HW?
So why are servo drives and VFD so different, even in the outer case and look? There are surely some firmware features, not strictly related to torque/speed/position control, that make them different, but couldn't them be implemented in both cases?