This scenario was discussed at a PPTQ I was judging at between me and the head judge and it posed a situation we both felt hard to answer.

Active player has a Wooded Foothills in play and 5 cards in hand. Player taps Wooded Foothills pays 1 life and searches for a Mountain. They shuffle their deck present to their opponent for cut and go to resume the game and discover they have accidentally shuffled the 5 cards from their hand into their library. Calls for judge.

How should this situation be handled?

This is actually a very straightforward GRV. Give the active player a penalty, decide whether one of the partial fixes applies (it doesn't), decide whether it's OK to backup (giving the player - essentially - 5 Demonic Tutors, it not). Therefore the game continues with the current state.

The player is down 5 cards, but has hopefully learned his lesson.

Every time this comes up it feels terrible for all involved, but Jasper's ruling is correct. Unfortunately, the player has just Mind Twisted themselves. Remind the player to be more careful.