Quantcast
Channel: Backup and restore
Viewing all 6664 articles
Browse latest View live

Automated backups unfinished

$
0
0
by Bernat Martinez.  

Hi everyone, recently I'm having a problem with a automated backups in Moodle. When I try to backup manually all courses, cerminal shows this error,

Server Time: Mon, 17 Aug 2020 11:09:46 +0200

Execute adhoc task: core\task\course_backup_task

... started 11:09:46. Current memory use 35.2MB.

Processing automated backup for course: Física.1BA(BO)

Automated backup for course: XXXXX encounters an error.

Exception: No se puede encontrar registro de datos en la tabla backup_courses de la base de datos.

Debug: SELECT * FROM {backup_courses} WHERE courseid = ? AND laststatus = ?

[array (

  0 => '165',

  1 => 6,

)]

Has anyone have the same problem or know the solution?


Thanks in advance


Re: Automated backups unfinished

$
0
0
by Bernat Martinez.  

In addition this error also appears...

Processing asynchronous backup for backup: b33367afd3ca83e879629c301ecd5576
!!! Detectado un error de codificación, debe ser corregido por un programador: A lock was created but not released at:
[dirroot]/lib/cronlib.php on line 154

Code should look like:

$factory = \core\lock\lock_config::get_lock_factory('type');
$lock = $factory->get_lock(cf236410d4e9e4b6d71e66552b20ce7ec02463d2);
$lock->release(); // Locks must ALWAYS be released like this.

!!!
Potential coding error - existing temptables found when disposing database. Must be dropped!

Re: Automated backups unfinished

$
0
0
by Bernat Martinez.  

Error code: codingerror !!
!! Stack trace: * line 117 of /lib/classes/lock/lock.php: coding_exception thrown
* line 76 of /lib/cronlib.php: call to core\lock\lock->__destruct()
* line 79 of /admin/cli/cron.php: call to cron_run()
!!
Potential coding error - existing temptables found when disposing database. Must be dropped!
PHP Fatal error: Uncaught Error: Call to a member function real_escape_string() on null in /home/benedu/public_html/lib/dml/mysqli_native_moodle_database.php:1140
Stack trace:
#0 /home/benedu/public_html/lib/dml/mysqli_native_moodle_database.php(1269): mysqli_native_moodle_database->emulate_bound_params()
#1 /home/benedu/public_html/lib/dml/moodle_database.php(1624): mysqli_native_moodle_database->get_records_sql()
#2 /home/benedu/public_html/lib/classes/lock/mysql_lock_factory.php(158): moodle_database->get_record_sql()
#3 /home/benedu/public_html/lib/classes/lock/lock.php(102): core\lock\mysql_lock_factory->release_lock()
#4 /home/benedu/public_html/lib/classes/lock/lock.php(116): core\lock\lock->release()
#5 [internal function]: core\lock\lock->__destruct()
#6 {main}
thrown in /home/benedu/public_html/lib/dml/mysqli_native_moodle_database.php on line 1140

Fatal error: Uncaught Error: Call to a member function real_escape_string() on null in /home/benedu/public_html/lib/dml/mysqli_native_moodle_database.php:1140
Stack trace:
#0 /home/benedu/public_html/lib/dml/mysqli_native_moodle_database.php(1269): mysqli_native_moodle_database->emulate_bound_params()
#1 /home/benedu/public_html/lib/dml/moodle_database.php(1624): mysqli_native_moodle_database->get_records_sql()
#2 /home/benedu/public_html/lib/classes/lock/mysql_lock_factory.php(158): moodle_database->get_record_sql()
#3 /home/benedu/public_html/lib/classes/lock/lock.php(102): core\lock\mysql_lock_factory->release_lock()
#4 /home/benedu/public_html/lib/classes/lock/lock.php(116): core\lock\lock->release()
#5 [internal function]: core\lock\lock->__destruct()
#6 {main}
thrown in /home/benedu/public_html/lib/dml/mysqli_native_moodle_database.php on line 1140

Re: Can I backup a course from Mysql and restore to Postgres?

$
0
0
by Dave Balch.  

The database any particular Moodle site uses will not affect .mbz backups, which are designed for reliable transfer of courses between Moodles .

Automatic Backup fails and manual backup does not complete

$
0
0
by Alexander Gres.  

Hello,

we have a Moodle installation with about 800 courses and the automatic backup continually fails to create a backup for two of our courses. If I try to create a backup of these courses manually, the process gets stuck at 2.18 seconds – 99.36% and no error messages are shown, even if I set the debug messages (debug) to developer and enable the display debug messages option (debugdisplay). The system status page shows that everything is ok. We are using version 3.9.1 (Build: 20200713) (2020061501).

The php log file shows no messages either. Can someone point me to a log file or similar that might contain something useful?

Thank you!

With best regards
Alex


Re: Automatic Backup fails and manual backup does not complete

$
0
0
by Ken Task.  

The very last step in autobackups is to *copy* the built .mbz file in moodledata/temp/backup/sometempdirectorythatlookslikeahash

to the destintation chosen in config ... destination default is in the sea of files /moodledata/filedir/

If it's just 2 or 3 courses, could be those course backups (if they are indeed built) have gotten so large the *copy* command used to move the built backup.mbz file in the temp build area just times out.

So to see ...

cd /path/to/moodledata/temp/backup/

ls

successful backups leave a 0 byte .log file.  Unsuccessful backups leave a .log file with something in it.   Those are text files and can be viewed with cat.

What you are looking for, however, are the directories with long names ... change into one of those and ls

If you see a moodle_backup.xml and a backup.mbz file ... guess is it timed out on copy to destination ... the backup.mbz file is a valid backup and can be manually moved to another destination .. not moodledata/filedir/

If you don't see moodle_backup.xml nor the backup.mbz file, nothing in there can be used and should be manually removed but before you do, begin the painful discovery of why.

There should be .xml files that match a directory ... example: files.xml and files directory.

Can't tell you anything specific at this point about poking around in there ... much depends upon what was in the course.

Hopefully your issue isn't running out of space ... but that could also be a cause.

df -h

Sorry ... no fix ....

On one system I had to give up auto backups due to a course reaching 130Gig - could back it up using backup.php script in admin/cli/ with a nohup (it took a couple of hours).

Also had to do a looping bash shell script through course id's and used the cli backup.php script .... had one script for large courses that used the nohup option.

'SoS', Ken

Re: Automatic Backup fails and manual backup does not complete

$
0
0
by Alexander Gres.  

I’ve found a folder and a corresponding logfile, however it does not really contain anything helpful to the current problem. The folder also contains a .mbz file at around 850MB, so the size should not really be a problem, as our backup target directory contains a couple of multi-gigabyte mbz files. I had a look at the .xml files too, however I was unable to find anything there either.

During the manual backup I noticed that I can specify what the backup should contain and what not. I guess I’ll just start disabling things in a binary search like fashion. Maybe I can pinpoint what causes the issue.

Regards,
Alex

Re: Automatic Backup fails and manual backup does not complete

$
0
0
by Ken Task.  


850M isn't a Gig!   So yeah ... not that large.  800 courses ... a lot to do.

And, normally, if one sees 800 in the .log file it finished with the actual backup build.

Then am gonna guess the backup script timed out at the *copy* stage where it copies the backup.mbz file there to desination and changes it's name according to preferences in backup options .... if there is a backup.mbz file there and it's valid ... that's the very last step.

Going back to GUI manual selecting modules to exclude will reduce the size of the course ... or should ... thus should also take less time to *copy* from build area to destination.

What are php settings for time for a script to run and memory a script can use?

Also check your task list for possible conflicts between what is probably a very long lasting autobackup process with any 'clean up' of temp areas.

How long does the autobackup process take?   Something to check into for determining conflicts with cron/task.  If it starts at midnight and doesn't finish before prime time for your site then ... also ... have to remember autobackup is really background ... server still has to contend with whatever else it's being asked to do.

'SoS', Ken



Re: Can I backup a course from Mysql and restore to Postgres?

$
0
0
by Colin Fraser.  

Sorry, was in a hurry and just coming from a conversation on a similar topic, so misread the question. Dave has responded to what you asked. Thank's Dave....

Re: Course copy fails, backup fails

$
0
0
by Brendon Hatcher.  

Update....

I created a new Virtual Machine.
I installed a fresh Moodle on this.
Did a backup successfully.

Then migrated my live Moodle to this VM.
Backups still fail in the same way.

Any suggestions?

Thanks
Brendon

Re: Course copy fails, backup fails

$
0
0
by Brendon Hatcher.  

My backups were failing because I had asynchronous backups enabled.

Course copying is still failing.

Re: Course copy fails, backup fails

$
0
0
by Brendon Hatcher.  

The copy course function still doesn't work, but I am able to restore the course and enter a new course name to create a copy.

[3.9] Automated backups consumes all disk space and trashdir is full

$
0
0
by Marius S.  

Hello,

We have updated our Moodle from 3.7 to 3.9.1
We have automated backups run by crontab command every Saturday morning and it finishes in Sunday afternoon/evening.
We keep 4 weeks backups in Moodledata and in specific directory (.mbz files), we skip courses not modified for 30 days.
So basically every week we make ~2200 backups and it's ~240GB.
But as we keep 4 weeks backups so basically oldest 2200 backups (~240GB) is deleted. As I understand moved to moodledata/trashdir and scheduled task should delete it from trashdir?
We have this setup for years. Currently ~600GB free disk space and this happened second time after Moodle update.

Problem:
This Saturday morning automatic backups started and Sunday afternoon we couldn't login to Moodle (Database reading error), so I checked server via SSH it was 0% space left on disk, then I checked size of moodledata/trashdir it was 600GB there..
In Filezilla I saw that moodledata/trashdir folder has 256 directories. Which is strange to me, because it seems that files are not being deleted fast enough?

We run our cron every 60 minutes.
In scheduled tasks "Cleanup files in trash  \core\task\file_trash_cleanup_task" runs every 6 hours by default, and from the logs I see that it was running without problems on Saturday and Sunday.
As I understand this task should delete files from moodledata/trashdir?

So why files are not deleted from trashdir and why it takes so much disk space?



Because as I said we also make .mbz backup files to specific folder (separate disk), so there is no change in that disk space. There is no huge swings in .mbz file size, maybe few GB.
Last week  was made 2254 .mbz backup files and 245GB, and two weeks ago it was 2206 files and 236GB

So for some reason when automatic backups starts Moodle put a lot of files to trashdir but file_trash_cleanup_task is not capable deleting so many files at a time?

Thanks,
Marius.

Re: [3.9] Automated backups consumes all disk space and trashdir is full

$
0
0
by Ken Task.  

An area to check config for:
Administration -> Server -> Tasks -> Task processing

What does
Site administration -> Server -> Tasks -> Task logs
when filter set to 'auto' and drop down set to Success show?


Compare \core\task\automated_backup_task with \core\task\backup_cleanup_task
is clean up romping on backups?

backups will use moodledata/temp/backup/

What, if anything, is in that directory or than 9 byte .log files?
Are there hash named directories and inside those a backup.mbz with
all the xml's and related directories?

There has always been a hard coded number of days to wait before removing anything from moodledata/trashdir/

"the draft files are deleted from the files table after 4 days, the trashed files (not present in files table) are deleted after 24 hours. This is intentional."
From https://tracker.moodle.org/browse/MDL-25592

Also remember that moodledata total space usage is affected by other modules/actions/activities ... like assignments and annotation/grading of assignments - this to say with a space crunch other areas might be involved ... like a teacher that decides to make their own backups of courses ... goes to user private files which admin level users cannot see.

In a space crunch ... like running out of space ... one can manual remove any/all files folders from moodledata/trashdir/ without issues.

'SoS', Ken

Re: [3.9] Automated backups consumes all disk space and trashdir is full

$
0
0
by Marius S.  

Hello Ken,

Administration -> Server -> Tasks -> Task processing all values are set to defaults. Scheduled task concurrency limit: 3, Scheduled task runner lifetime: 30min,  Ad hoc task concurrency limit: 3,  Ad hoc task runner lifetime: 30min.

If I go to Site administration -> Server -> Tasks -> Task logs and search for automated_backup_task Success and backup_cleanup_task Success
For both I get 24 lines and the Start time is almost the same, in some lines time is different in 1 minutes, but mostly it's the same Start time.

In moodledata/temp/backup currently there is 3500+ .log files from dates Aug 17th to Aug 24th. But all files are 0 byte size.
Also there is 12 directories with dates Aug 17 & Aug 23. Inside these directories there is xml files, and other folders like sections,course,files,activities.

Yes, I understand that moodledata space usage if affected by assignments, teachers uploading new files/deleting old ones etc. But this is still strange because this never happened before and we had like 200-300GB free space and we didn't face this problem. Now we have 600GB free space and we ran out of space two times and both times was during weekly course backups in the weekend.
Semester is not even started so there is not much action in the Moodle system right now, so I highly doubt about teachers taking backups, uploading tons of files, etc.

Today in the morning weekly backups finished (as I deleted files manually from moodledata/trashdir so automated backups continued and finished this morning). So I checked on our Specific location where additional .mbz backups are create, so for this week it was 2370 courses and total .mbz backups size 248GB.
So I suppose in Moodledata backups takes similar size and oldest backups are deleted when new are created, so there should be no problems.. Maybe in Moodle 3.9 something changed in backups/deletion process?

Thanks,
Marius.

Re: [3.9] Automated backups consumes all disk space and trashdir is full

$
0
0
by Ken Task.  

"For both I get 24 lines and the Start time is almost the same, in some lines time is different in 1 minutes, but mostly it's the same Start time."

Think about that ... same start time?

"Also there is 12 directories with dates Aug 17 & Aug 23. Inside these directories there is xml files, and other folders like sections,course,files,activities."

Where there any backup.mbz files in those directories?

If so how large were they? 

If there wasn't a backup.mbz nor a moodle_backup.xml then that course backup failed for some reason.   View moodle_backup..xml file to see course name then go visit that course to see what's in it.  Those can be manually erased without consequence.

"Semester is not even started so there is not much action in the Moodle system right now, so I highly doubt about teachers taking backups, uploading tons of files, etc."

Seems to me that would be a very good time for teachers to be active in their courses ... including making a backup.   But ... only you would know that about your own site ... if you'd been monitoring and have some historical info.

That would also mean that courses haven't changed .. right?   Why would you need another backup of that course if it hadn't changed?

Is the finish time consistent?

Believe you said your cron set to run every 60 minutes?   Think the recommendation is to run it once every minute now.  Everything that cron does won't be executed at each run ... task list.

You might also consider increasing those time outs ... slightly higher.

'SoS', Ken

Re: Can I backup a course from Mysql and restore to Postgres?

$
0
0
by Visvanath Ratnaweera.  

There is Site administration > Development > Experimental > Database migration (in rel 3.9 at least):
Never tried though. Any experience?

Re: [3.9] Automated backups consumes all disk space and trashdir is full

$
0
0
by Marius S.  

Compare \core\task\automated_backup_task with \core\task\backup_cleanup_task:
Tasks

These course backups in /moodedata/temp/backup/ maybe failed because we ran out of space. At the moment I see only 4 directories, so other 8 courses was completed or was deleted automatically. For these 4 remaining courses only two had moodle_backup.xml file and checked these course, they had automated backups done last week. The other ones maybe had to be course backups but didn't started because run out of space? Just my guess...
Currently /moodledata/temp/backup/ is only ~300MB.

I would like to get back to /moodledata/trashdir/ then.. So now in Filezilla I see that it has 256 directories.
And this is the maximum directories size which can be as I understand..
Trashdir

And I am sure that these directories at 4-5AM for sure are not teachers who deleted something, so these are the oldest backups files from moodledata which was deleted because new was created. So even if this is old backup it will stay in trashdir for like 24hours or so?

Or if we would run our cron.php every minute it would get deleted much faster?

Also about which timeouts are you referring to?

Thanks,
Marius.

Re: [3.9] Automated backups consumes all disk space and trashdir is full

$
0
0
by Richard van Iwaarden.  

Marius, I have had the exact same problem a couple of times with different Moodle versions. The problem has always been the recycle bin, not the automated backup. What I do now is 1) disable the recycle bin and 2) delete everything from the temp backup folder.

After a few days I turn the recycle bin back on and the problem seems solved for some reason.

I know this is a bad solution and many other people will find this problem (it has been reported several of times) but it solves it for now.

Re: [3.9] Automated backups consumes all disk space and trashdir is full

$
0
0
by Marius S.  

Thanks Richard, but if you disable recycle bin and later enable it, so all content which teachers deleted will be lost?
Because recycle bin is enabled in our Moodle and we set to keep files for 30 days, so if teacher deletes something we don't need to restore all course/item from backup file, he can find his deletes files in recycle bin for 30 days.

Marius.
Viewing all 6664 articles
Browse latest View live


<script src="https://jsc.adskeeper.com/r/s/rssing.com.1596347.js" async> </script>