Looks like Linux (Ubuntu) and you have command line access .... Good!
How much disk space do you have free?
Let's try taking Apache out of the loop with backup.
You can safely remove all files folders in moodledata/temp/backup/
1. Get the course ID of the course you cannot backup.
2. cd /pathtomoodlecode/admin/cli/
3. php ./backup.php --courseid=# --destination=/somedirectory/wherethereisplentyofspace/
In another terminal session, go to your data directory/temp/backup/
When you begin the php script to backup the course id, look in that other terminal session window
There will be a contenthashed named directory created ...
cd thatcontenthashednameddirectory
type: watch "ls -l;df"
and you can see what Moodle is working on ...
Can tell you right now files are numerous from what you show above that will take the longest (largest subdirectory/largest .xml file). So when the backup routine hits the part for files, it will slow down and take time ... even the command line might appear to stop.
BTW, the 'course' directory isn't really the course!
Wonder what another terminal session running top would show?
'spirit of sharing', Ken