only use your map
ignore the quest arrow.
The people don't tell you specific directions but they mark it on your map.
If you ignore the compass completely it all makes sense.
Actually, this makes a lot of sense.
They mark it on your map, and thats (in my opinion) better than giving you a list of directions.
Ive always been bad and getting to places without a map, and thats part of the reason Im not a huge fan of Morrowind.