Confusing ... to say the least, however ...
How much memory does server have? Is it an all-in-one server where DB server and Web server on same machine?
Check your moodledata/temp/backup/ directory. That is used to un-compress mbz files (backups). Directories with long names are created. Inside those, look for large .xml files ... files.xml works with files directory contained therein, questions.xml ... etc. If there is a moodle_backup.xml that's entire road map for restoring a course.
At moodledata/temp/backup/ there could also be 'longfilename'.log files ... 0 byte .log files are successful restores. Any .log file larger than 0 bytes contains info about the restore process ... inspect those. Moodle has to un-compress the .mbz and build a plan to restore. The .log files show at what stage the restore reached.
Also check web service error logs and if you are running antivirus ... check anything that might through as well.
Restores can push server limits ... so even if things look right, consider increasing php limits time for a script to run, max memory used for a script. If one finds 'MySQL Server has gone away' in web service error logs, it's max_allowed_packets that needs to be increased. There is also nuimber of opened files variable in DB config. Large courses might require more file be opened.
Have found MySQLTuner to be a good tool for helping config the DB server.
'SoS', Ken