Work on raising your strength and smithing skill. How fast you mine is determined by strength * tool ql, and so far everyone's guess is that there's just a random requirement to mine places.
Ex, You can mine in your cave. You make a minehole. You can no longer mine because this layer/part of the layer needs a higher str * tool ql.