Are you confident that the Moodle you are restoring to has all the same question types in it that the machine the backup was made on has?

Thanks a lot, Usman, for all your help.

It is very interesting that when I installed PHP 7.0.2 on my Moodle 2.9.1 I can see certain pages OK but certain pages are simply look blank (nothing is displayed). That is why I suspect that some plugins that I use are incompatible with PHP 7. Unfortunately, backup is one of these blank pages.

Take care,

Eugene
PS Also, I've messed with php scripts on some pages to make our Moodle more dialogic, democratic, and collaborative for our classes, so it will be a lot of "fun" to adjust these scripts to Moodle 3.x and PHP 7 in the summer for me . But the backup php page is not affected by my modified scripts, so it should be plugins on that page that are incompatible with PHP 7...

Sounds like you do have terminal/ssh access.   So do this:

In the Moodle UI get to the screen where you click the button to begin the process and pause.

In terminal/ssh session, login as root and go to the moodledata/temp/backup/ directory.

Remove anything in there first so we can see what we are working with.   This is safe to do.

From moodledata/temp/backup/ location: rm -fR * [ENTER]  MAKE SURE YOU ARE LOCATED THERE!

Back to the Moodle UI window in the browser - click the button.

Back to the terminal ... do an ls

You'll see a new directory by a long name like 54asdlghqwer;lqwerhi34qwerghsls

cd 54[tab key] which should find the entire long directory name.

Once inside that directory ... this is where moodle builds the backup ... issue the following command:

watch "ls -lR;df"

This runs watch ... which executes both ls -lR in one window and displays how much diskspace is remaining in the second window.  It refreshes every 2 seconds by default.

Fairly quickly you will see a 'activities' directory created ... quit the watch command, then reissue it wtih:

watch "ls -lR ./activities/ ;df"

You'll see quiz directories being created by their quiz ID number.

Now IF/when the Moodle UI kicks out the error, the watch window will show where it stopped.

If you note the quizIDnumber directory last created ... you mght be able to then deduce the quiz it was trying to build - and thus, the next time you do this, EXCLUDE just that one quiz.

Yeah, it's involved ... but Moodle UI doesn't provide much help other than a log that shows some numbers ... like 800 ... which, BTW, are tied to what the backup/restore process is currently building, etc.. Think 800 is activities which includes not only quizzes but other potentially heavy data stuff like questionnaires..

The error about the temptable is somewhat misleading, BTW ... there is NO temp table remaining after a failure like this cause the session established with the DB to do the backup process is killed and when MySQL discovers that, the session is ended ... ending sessions auto removes temp tables in MySQL.

Ok ... you offered a look see at the course ... PM me with the URL where I can obtain the backup and I'll load it up on a server to which I have access where I can inspect the backup and try it out there.  Is it no user?

'spirit of sharing' ... and going the extra mile! :|

Ken

Yes.  My first attempts were all backing it up and restoring within the same site.  I've also tried on my dev site on the same server with identical plugins (a few slightly older versions though) and a brand new clean/no plugin version also. 

Our new semester workflow is:

• Move all the old used courses to an Archived Courses category and hide them from students.
• Use backup to make a copy of the old course without any user data.
• Restore the backup to make the new fresh clean course that can be edited away.

Link sent for the course.

I do have ssh access.  I'll work on watching the restore run as soon as I have a chance.

Results of restore to another server ... at bottom ... possible solution ...

Server upon which it restored which might be a factor in this case (course was only 80M - which, in my book, isn't all that large!):

el5 x86_64
16 Gig memory
Apache as mod (not fastcgi)
max_input_vars 1500
memory_limit 256M

Might have to increase values for execution time, etc.. in php.

Does finish (takes a very long time) with this error reported:

Aliases restore failuresHelp with Aliases restore failures
Some aliases included in the backup file could not be restored. The following list contains their expected location and the source file they were referring to at the original site.

    /2331/mod_resource/content/0/Academic Integrity.pdf
    /2337/mod_resource/content/0/MLA Reference Sheet.pdf
    /2343/mod_resource/content/0/Adding Buttons Tutorial.pdf

more to listing ... probably every link to a file in the course.

and lower down in the technical information and warnings:

processing file aliases queue
  looking for repository instance by type filesystem

A BUNCH of lines like these:

  unable to restore alias (unable to find a matching repository instance) /2331/mod_resource/content/0/Academic Integrity.pdf
  unable to restore alias (unable to find a matching repository instance) /2337/mod_resource/content/0/MLA Reference Sheet.pdf

Work around to restore this might be:

Replicate the file system repo you had on the production server where this backup
was made on the dev server or the server to which you are migrating and then
the aliases should/might work.

Thanks ... you've taught me not to use aliases to file system repo's IF one wants the course to be 'transportable' to other systems that might not have/want file system repos.

Have sent to Joshua the link to restored course .. he can inspect what's missing.  PM me when you desire the course/account there to be gone! ;)

'spirit of sharing', Ken

Since you have ssh access you might consider setting up a 'unirepo' (universalrepo) using file system (you're already using file system repo with your courses ... the alias problem when restoring to other server).  

Setting up a unirepo would enable you to do command backup of a course in production server ... which would save the backup to the unirepo.   Those backups, run from the command line, take apache out of the loop ... will be faster and can backup larger courses.

Then, in the dev instance, set up a file system repo pointed to the unirepo.   Set up the file system repo in only one course that you won't be deleting in the dev instance, and you can backup/restore on that server in less time ... no download only to have to turn around and upload (the backup .mbz files).

I've done exactly as briefly described above for 3 sites (different entities) who 'share' courses amongst themselves - one unirepo - 3 production sites ... no issues with time outs uploading/downloading, no increase file size uploads, etc.

Can share a how to if interested.

'spirit of sharing', Ken

After Ken managed to get it restore on his server, I started assuming that it was a problem with my configuration.  I think it's likely I'm hitting a timeout somewhere along the way but I'm not sure where.  We're using mod_php and I increased the max_execution_time and max_input_time far longer than it has taken to fail in the past.  I also increase max_input_vars and memory_limit just to see what happened.  All of those changes had no effect.

Rather than continue to beat my head against a wall there not knowing which direction to look at, I tried a different approach and downloaded Moosh.  I figured this might work since there wouldn't be timeout issues here.  Restored the course via the command line and its up and running.  Moosh FTW.

As far as the course size goes:  The backup isn't huge because I've aliased so many files in the past.  Also, that was a backup without student data.  The backup at the end of the semester with student data is 450MBish.  That and anything that takes 15-20minutes to restore counts as large in my book.  Most of that seems to be in the question bank...

As far as the aliases go: This course started out life on a 1.9 server, changed sites, and then went to 2.1 and then up one version at a time.  I, like many other people, severely missed the convenience of updating files via FTP in 1.9 and set up the file system repository so that I could still do that.  Then a version or two later came drag and drop file uploads and I was again irritated that I did all that work aliasing files only to have a much better option a year later.  Since I'm not currently planning to move my course anywhere then I've just been lazy and not redone all the aliased files.  I'll update them all at some point.  Of course I think there are still files in the Legacy Course Files that get used in quiz questions.  In all my spare time...

So I've accomplished my task but I'm still happy to take suggestions on other places to look to try to fix the fact that I still can't restore the course the normal way.

I see what you're suggesting here.  That's a really slick way of doing that.  You ought to write that up somewhere in the docs.  That would have been helpful last night when I was bouncing between 3 different sites on the same server.

Can't restore this one course ... but others are possible.

The course in question is still in legacy files (it restored that way).   So that means it started as a 1.9 or earlier ....  that's also a factor.  One of these days, an update/upgrade to Moodle will announce ... oh, we no longer support Legacy files. So, one piece of advice, even if you can't do it right now ... work on it ... do it soon ... get the course out of legacy.   Use a file system repo to link to files ... but don't alias to them! ;)

Reading the history, it could be that the question bank is in need of repair.   Besides files, that's the other heavy processing point in backups and restores.

Did you check out the quizzes on the server where I was able to restore it?

7.5M    ./questions-DemonChem 1516F-Gen Chem-20160119-2321.xml

On your server, login as admin, then using the URL line, go to:

http://yoursite/admin/tool/health/

It will probably find some errors related to quizzes and provide a 'proposed solution' button/box which is an SQL query one copies and paste into an app that can do MySQL ... uhhhh, command line at the 'mysql>' prompt works also.

Will that fix this course?  Well, at this point, I'd say anything would help.

Dunno if you will see anything ... on the server to which I restored, no health check errors ... I expected to see some related to quizzes.   On your server, however, you might as it does contain other courses and other banks.

Moosh ... yes, knew about that but hadn't really needed it ... yet.   Good to know that Moosh worked with your troubled course.

Do you need the course I restored?   I need to remove that which is not used. ;)

'spirit of sharing', Ken

If your courses are in Legacy, they will forever be in legacy on backup and restore - unless one edited the moodle_backup.xml and files.xml in an un-archived backup, re-compressed (properly), then restore.

Maybe a slightly different flow is needed if there are other courses like this one.

At end of this year, backup the course for archival purposes ... could restore to dev using Moosh if need be.   I'd do that via command line backup from the script in admin/cli/ and point the backup to go to a unirepo that dev can reach/see.

Instead of backup and restore, do 'reset' of the current course.   Choose options to remove students and their assignments, quiz attempts, grades ... anything related to student data ... basically roll it back to like it was before students were enrolled.   Backup the course again via command line.   That's a base line course then one could work on in spare time to rid of Legacy, etc., etc...

Also consider breaking up a full year course that you know by the end of the year reaches 'critical mass' when it comes to backups/restores.   Make them semester courses even though the real class is a full year.  I've had to do that with courses like 'Digital Photography' and "American Sign Language" (the second used video for everything!   The best course I've ever seen when considering objectives of the course and appropriate use of technology!  Students had a blast with it and learned along the way!!!).

Anyway ... 2 cents worth ...

'spirit of sharing', Ken

And please let me know if you need that course I restored on the other box.  I need to remove it.

Good day !

After restore course I've not find my file in restored course. After restore I get following messges:

missing file (картинка.jpg) contenthash in backup for component course
missing file () contenthash in backup for component mod_resource
missing file () contenthash in backup for component mod_resource
missing file () contenthash in backup for component mod_resource
missing file () contenthash in backup for component mod_resource
missing file () contenthash in backup for component mod_resource
missing file () contenthash in backup for component mod_resource
missing file (НОТ - копия.ppt) contenthash in backup for component mod_resource
missing file () contenthash in backup for component mod_resource
missing file () contenthash in backup for component mod_resource
missing file (система 5 S.mp4) contenthash in backup for component mod_resource
missing file () contenthash in backup for component mod_resource
missing file (Задачи - копия.docx) contenthash in backup for component mod_resource
missing file (Нормирование.mp4) contenthash in backup for component mod_resource
missing file () contenthash in backup for component mod_resource
missing file () contenthash in backup for component mod_resource
missing file () contenthash in backup for component mod_resource
missing file () contenthash in backup for component mod_resource
missing file () contenthash in backup for component mod_resource
missing file () contenthash in backup for component mod_resource
missing file () contenthash in backup for component mod_resource
missing file () contenthash in backup for component mod_resource
missing file () contenthash in backup for component mod_resource
missing file () contenthash in backup for component mod_resource
missing file () contenthash in backup for component mod_resource
missing file () contenthash in backup for component mod_resource
missing file () contenthash in backup for component mod_resource
missing file () contenthash in backup for component mod_resource
missing file (Бестарифная.ppt) contenthash in backup for component mod_resource
missing file (Премирование.mp4) contenthash in backup for component mod_resource
missing file () contenthash in backup for component mod_resource
missing file () contenthash in backup for component mod_resource
missing file () contenthash in backup for component mod_resource
missing file () contenthash in backup for component mod_resource
missing file () contenthash in backup for component mod_resource
missing file () contenthash in backup for component mod_resource
missing file () contenthash in backup for component mod_resource
missing file (Персонал.mp4) contenthash in backup for component mod_resource
missing file () contenthash in backup for component mod_resource
missing file () contenthash in backup for component mod_resource
missing file (ФЗП - копия.ppt) contenthash in backup for component mod_resource
missing file () contenthash in backup for component mod_resource
missing file () contenthash in backup for component mod_resource
missing file () contenthash in backup for component mod_resource
missing file () contenthash in backup for component mod_resource
missing file () contenthash in backup for component mod_resource
missing file () contenthash in backup for component mod_resource
missing file () contenthash in backup for component mod_resource
missing file () contenthash in backup for component mod_resource
missing file () contenthash in backup for component mod_resource
missing file () contenthash in backup for component mod_resource
missing file () contenthash in backup for component mod_resource
Module not fully restored: id 4411
Module not fully restored: id 4412
Module not fully restored: id 4415
Module not fully restored: id 4419
Module not fully restored: id 4421
Module not fully restored: id 4424
Module not fully restored: id 4425
Module not fully restored: id 4426
Module not fully restored: id 4429
Module not fully restored: id 4432
Module not fully restored: id 4434
Module not fully restored: id 4435
Module not fully restored: id 4437
Module not fully restored: id 4438
Module not fully restored: id 4439
Module not fully restored: id 4440
Module not fully restored: id 4441
Module not fully restored: id 4443
Module not fully restored: id 4445
Module not fully restored: id 4447
Module not fully restored: id 4448
Module not fully restored: id 4451
Module not fully restored: id 4452
Module not fully restored: id 4453
Module not fully restored: id 4455
Module not fully restored: id 4457
Module not fully restored: id 4458
Module not fully restored: id 4461
Module not fully restored: id 4462
Module not fully restored: id 4463
Module not fully restored: id 4464
Module not fully restored: id 4465
Module not fully restored: id 4466
Module not fully restored: id 4470
Module not fully restored: id 4471
Module not fully restored: id 4472
Module not fully restored: id 4473
Module not fully restored: id 4474
Module not fully restored: id 4475
Module not fully restored: id 4478
Module not fully restored: id 4479
Module not fully restored: id 4480
Module not fully restored: id 4481
Module not fully restored: id 4482
Module not fully restored: id 4485
Module not fully restored: id 4486
Module not fully restored: id 4488
Module not fully restored: id 4491
Module not fully restored: id 4492
Module not fully restored: id 4493
Module not fully restored: id 4494
Module not fully restored: id 4495
Module not fully restored: id 4497
Module not fully restored: id 4499
Module not fully restored: id 4500
Module not fully restored: id 4501
Module not fully restored: id 4502
Module not fully restored: id 4506
Module not fully restored: id 4507
Module not fully restored: id 4511
Module not fully restored: id 4512
Module not fully restored: id 4515
Module not fully restored: id 4517
Module not fully restored: id 4518
Module not fully restored: id 4521
Module not fully restored: id 4522
Module not fully restored: id 4523
Module not fully restored: id 4524
Module not fully restored: id 4525
Module not fully restored: id 4528

What is the problem, I can't uderstand. 

Please help me solve problem, asap. I need restre some course.

The restore obviously did not restore correctly.  I would try again.  Are you restoring a course from the same site or another site?  What versions of moodle?

I've try restore in the same site but as new course. 

Moodle 2.7.11+ (Build: 20151119)

Thanks in advance ! 

So, as I suggested, delete that course.  Go back and start again.  New backup and new restore.  I would also suggest turning on debugging while you are restoring to see if it throws some errors.

I think you can safely delete the course that you restored now.  I really appreciate you confirming that it could be restored.

I'm pretty sure Moosh worked simply because it ran from the command line and thus avoided any time out problems.  I need to look into where this timeout issue is coming from.  The fact that I'm not getting any useful messages on what's happening means I'm at a loss...

I'll take a look soon at the health tool and see what comes up.

Removed course and account.

When removing the course ... error ... had to increase the php time to run from default 60 to 120 (max_execution_time) to fully delete the course.   Deleting a course takes less time than restoring one so would suggest a start point for settings related to time outs be set to at least 120+.

'spirit of sharing', Ken

I have same problem...

So I dont have a complete fix but I changed my file and directory permissions. I was using the moodle recommended secure permissions found here.

 https://docs.moodle.org/30/en/Security_recommendations

 I then changed my phprc file like so. 

------------------------

post_max_size = 105M

max_execution_time = 500

max_input_time = 500

memory_limit = 1024M

fastcgi_read_timeout 300

request_terminate_timeout = 300

log_errors = 1

error_log = /home/dreammoodle/php.log

-----------------------

Im now able to create users but deleting a course sometimes fails. Also restoring a course thats over 30MB seems to always fail. I made sure my site policy is allowing the php default of 100MB but still no luck. 

Still nothing in the apache error log. Once I get the 500 error when restoring a course, I refresh then try restoring it again and I get this error. 

error/error_question_answers_missing_in_db

Which just tells me that I need to add my questions manually. 

Hello,

I have been struggling with the automated backups for a few weeks.

Automated Backups and Manual Backups were successfully completing before creating and enrolling users.

There are only 4 courses 3 of which are Quiz only and 200 users.

If I select "Include Users" backups fail. Backups also fail if I select only "Include Users".

I tried calling  "php ...\wwwroot\admin\cli\automated_backups.php" via powershell (times out after 1 hour), backup was started and hanged after a minute or so...

Folders and Log files are being created under moodledata\temp\backup.

There are no errors in PHP Error Log.

Any help will be appreciated.

Configuration:

• Moodle 3.0.2+
• Azure Web Apps
• Azure SQLDatabase
• PHP 5.6

Debug Log:

[22-Jan-2016 19:59:03 Europe/Istanbul] instantiating backup controller e8b1dad47ec2598af7fb687e57abc479
[22-Jan-2016 19:59:03 Europe/Istanbul] setting controller status to 100
[22-Jan-2016 19:59:04 Europe/Istanbul] loading controller plan
[22-Jan-2016 19:59:13 Europe/Istanbul] setting controller status to 300
[22-Jan-2016 19:59:13 Europe/Istanbul] applying plan defaults
[22-Jan-2016 19:59:13 Europe/Istanbul] setting controller status to 400
[22-Jan-2016 19:59:13 Europe/Istanbul] setting file inclusion to 1
[22-Jan-2016 19:59:13 Europe/Istanbul] checking plan security
[22-Jan-2016 19:59:13 Europe/Istanbul] setting controller status to 500
[22-Jan-2016 19:59:13 Europe/Istanbul] checking plan security
[22-Jan-2016 19:59:15 Europe/Istanbul] checking plan security
[22-Jan-2016 19:59:15 Europe/Istanbul] checking plan security
[22-Jan-2016 19:59:15 Europe/Istanbul] saving controller to db
[22-Jan-2016 19:59:15 Europe/Istanbul] calculating controller checksum 0a311e2aaec874b62d63676dbc33a82f
[22-Jan-2016 19:59:29 Europe/Istanbul] loading controller from db
[22-Jan-2016 19:59:29 Europe/Istanbul] checking plan security
[22-Jan-2016 19:59:31 Europe/Istanbul] checking plan security
[22-Jan-2016 19:59:31 Europe/Istanbul] setting controller status to 700
[22-Jan-2016 19:59:31 Europe/Istanbul] saving controller to db
[22-Jan-2016 19:59:31 Europe/Istanbul] calculating controller checksum 697adfce460ce258f51f21cc866437e8
[22-Jan-2016 19:59:36 Europe/Istanbul] loading controller from db
[22-Jan-2016 19:59:36 Europe/Istanbul] setting controller status to 800

Many Thanks