Well, I can see the log file under administration -> reports -> backup.
So I don't really need it mailed to me.
And yes, about 3000 courses in 2 hours. We have very fast servers I guess. Did you ever use the benchmark plugin for Moodle? It gives us very good results.
Files ... your moodledata/temp/backup/ - this I think you already know.
DB - Just tables ... have a look at them to assure yourself ... I've included the column info below:
mdl_backup_courses
id bigint(10) No Primary NULL auto_increment
courseid bigint(10) No None 0
laststarttime bigint(10) No None 0
lastendtime bigint(10) No None 0
laststatus varchar(1) No None 5
nextstarttime bigint(10) No None 0
mdl_backup_logs
id bigint(10) No Primary NULL auto_increment
backupid varchar(32) No Indexed
loglevel smallint(4) No None NULL
message longtext No None NULL
timecreated bigint(10) No None NULL
mdl_backup_controllers which logs each backup
id bigint(10) No Primary NULL auto_increment
backupid varchar(32) No None
operation varchar(20) No None backup
type varchar(10) No Indexed
itemid bigint(10) No None NULL
format varchar(20) No None
interactive smallint(4) No None NULL
purpose smallint(4) No None NULL
userid bigint(10) No Indexed NULL
status smallint(4) No None NULL
execution smallint(4) No None NULL
executiontime bigint(10) No None NULL
checksum varchar(32) No None
timecreated bigint(10) No None NULL
timemodified bigint(10) No None NULL
Truncating all of them results in Moodle running autobackups for the first time. So the settings for skip if not modified kinda things won't be known to Moodle and thus included.
With the number of courses you have would check that you have enough space to do this ... I would set preferences to save to an alternative directory (that you must manually create) and not to moodledata/filedir/
This clue: "WHM Cpanel and I think CentOS" indicates your provider might offer attached devices ... data drives. Providers like Rackspace and Google do offer customers such things ... those would have to be mounted by your moodle server and they would show up with a command like:
df -h
Example of a RackSpace box:
main drive ... OS, moodle code, etc:
/dev/xvda1 158G 38G 114G 25% /
attached data devices/drives. First one is Rackspace ... second is a Google Bucket ... note the sizes and how much available ... size, used, available
/dev/xvdb 99G 73G 22G 78% /mnt/data
sossrv 1.0P 0 1.0P 0% /mnt/gbucket
The Google bucket is 1 Pentobyte ... operating system command can't calculate how much is used or available, but I can tell you it will take me a lifetime to fill that up!
Is your entity adverse to using Google? Above is a CentOS 7 server hosted on Rackspace but have access to that Google bucket for backups.
Wouldn't use a Google Bucket for all of moodledata but sure is great for backups!
Amazon has similar/same.
'SoS', Ken
I have just upgraded to 3.9.2+ and am trying to back up some courses; one as a cartridge 1.1 thing (testing it out to see if it helped with another issue). However, I get an error every time...
Hi
I am fairly new to Moodle, particularly in troubleshooting scenarios and have run up against a wall trying to solve this issue. When I built the course for my client I did not realize she wanted to restrict access on a per-topic per-student basis and had created a single course with 14 topics - each with their own lessons in.
In order to break them up into 14 separate courses to allow her a greater degree of control over who can access which content, I tried to duplicate the course so I could remove 13 of the topics from it and be left with a perfect copy of the course with just 1 topic, and then repeat this process to end up with 14 courses each with a different topic in it.
However, the backup process never executed, and I am stuck with process pending which prohibits me from using any other kind of backup to get a copy of the course. I find today that I can do a second duplicate request, but it then also sits as process pending.
So the question is: How do I stop that process? Or how do I get that process to complete?
~ Ian.Continued for extra screenshots...
I felt I should come back to the top and add this disclaimer. I am always talking about automated backups. I know I sometimes refer to it as backup in my description of the issue, but please know that I always mean automated backup.
Experienced a hardware failure during an automated backup. After getting everything put back together, I cleared out the backup folder to get fresh course backups however all courses that were skipped during the last successful backup were not created in the post hardware failure backups even though they don’t exist.
The backup email digest sent before the hardware failure and the one sent after the server was operational again were identical:
Summary
==================================================
Courses: 281
OK: 124
Skipped: 157
Error: 0
Unfinished: 0
Warning: 0
Automated backup pending: 0
Backup completed successfully
I ran select * from moodle.mdl_backup_courses; in MySQL and found that every course that wouldn’t create a new backup had a “laststatus” of 3. Do I need to go through and change them to a different status to get them back on track? Or can I truncate the mdl_backup_courses table to get all fresh backups? Any of the other backup tables, mdl_backup_controllers or mdl_backup_logs, need attention?
I am not sure if this is a similar, but this user had a status stuck in Running and it was recommended to truncate the backup tables:
https://moodle.org/mod/forum/discuss.php?d=397529
and follow up here:
https://moodle.org/mod/forum/discuss.php?d=397529&parent=1603589
Thank you
Well, one could do 'exacto-knife' and change values in DB tables ... that's one way.
The other ... truncating all related tables ... moodle doesn't know automated backups have ever run ... like it was the first time. So preferences for course backups (system wide) come into play ... users/no users, events/logging ... how long ... plus, if were me, I'd make sure I had space outside of moodledata/filedir/ where I could see the backup files and their sizes. Got enough space for that?
Best way? Dunno! Works for me! ;)
'SoS', Ken
At the time I did this on a K12 server, the tables for tracking what courses etc. were messed up. About 3 references were next run dated for next year!!!! Well, that wouldn't do ... one of those courses used daily.
My issue happened to be a digital media course that just kept growing - first noticed when backup was approaching 100Gig! It finally reached 130Gig before I finally had to communicate with school admin and teacher.
Autobackups then broken up into 3 bash shell scripts and used admin/cli/backup.php with options ... small, medium, large ... the large had only 4 courses in it. Couldn't do it daily ... but close enough (day or 2 work lost if a backup had to be used to restore ... never needed).
Retirement ... hmmmm ... fraid am past the 'golden years' of that! We all grow old ... remember that! [Note: so does software!!!!]
Best of luck with your choice!
Hangin' round means I can spot what doesn't work many times cause I've been there done that!
'SoS', Ken
I'm running into a weird issue on two older Moodle 3.5 installs (I know - upgrades are pending), wherein the moodledata/temp/backups folder grows abnormally large very quickly (hundreds of gigs overnight), and Moodle doesn't appear to be cleaning up after itself. Cron appears to be running normally on each server.
The backups were originally configured as automated - however after this problem started happening, I switched them to manual so I could capture a log of what was happening during the backup process. But here's the weird part: despite the backups being configured as manual (and not configured to run via the OS cron until tomorrow), a full restart of both the webserver and PHP backends, *and* me disabling the automated backups task in scheduled tasks ... Moodle still appears to be running the scheduled backup task. Now.
Has anyone seen anything like this before? I'm thinking that there's an outstanding backup job still scheduled that Moodle is trying to complete. Any thoughts as to where I could look in the database to confirm and/or disable this?
The mdl_task_scheduled table does show the automated backup task as disabled:
+----+------------------------------+------------------------------------------------------+-------------+-------------+----------+--------+------+-----+-------+-----------+-----------+------------+----------+
| id | component | classname | lastruntime | nextruntime | blocking | minute | hour | day | month | dayofweek | faildelay | customised | disabled |
+----+------------------------------+------------------------------------------------------+-------------+-------------+----------+--------+------+-----+-------+-----------+-----------+------------+----------+
| 24 | moodle | \core\task\automated_backup_task | 1601034679 | 1601038200 | 0 | 50 | * | * | * | * | 0 | 1 | 1 |
Check what's in moodledata/temp/backup/ and also in trashdir
Then ... what happens if ...
cd code//admin/tool/task/cli/
Create a bash shell script called 'cleanup' which contains
php schedule_task.php --execute="\core\task\backup_cleanup_task";
php schedule_task.php --execute="\core\task\file_temp_cleanup_task";
php schedule_task.php --execute="\tool_recyclebin\task\cleanup_category_bin";
php schedule_task.php --execute="\tool_recyclebin\task\cleanup_course_bin";
then check moodledata/temp/backup/ and moodledata/trashdir/
Above won't fix ... more exploring what's there and might lead to other discoveries!
'SoS', Ken
MySQLServer has gone away .... too much data in a chunk.
max_allowed_packet needs to be increased in my.cnf config for DB server.
Query:
mysql> show variables like 'max_allowed_packet';
https://dev.mysql.com/doc/refman/5.7/en/program-variables.html
Think default is 8M (I think) ... wouldn't hurt for your server to set to 16M - higher if you continue to get 'gone away'.
trashdir ... you can manually remove contents ... no adverse affects
Delete stale temp files (core\task\file_temp_cleanup_task)
Whats in moodledata/temp/backup/
Successful backups leave 0 byte log files.
Any .log file larger than 0 bytes indicates a failed backup for some reason.
Unfortunately, the .log files aren't really that helpful in discovering why
as they show only at what point in the 'plan' (a number) it failed.
** But, if there are a bunch of directories by hash names, those are what moodle was using to build a backup file.
Some could actually have 'backup.mbz' files in them ... which means the backup was success but the process failed at the last step - copying the .mbz file to destination.
Anyhoo ... everything in moodledata/temp/backup/ could be manually removed.
Suggest it's time for you to install and run MySQLTuner. Will help with DB server config.
https://github.com/major/MySQLTuner-perl
For info about Tuner ...
https://www.linode.com/docs/databases/mysql/how-to-optimize-mysql-performance-using-mysqltuner/
There are also some command line only scripts in moodlecode/admin/cli/ that help with DB for Moodle ... all begin with mysql_ and all have 'help' screens as defaults when called via php mysql_scriptname.php
'SoS', Ken
I run Tuner on a regular basis. Tweaks have been needed from time to time ... and always upwards on sites that are used.
In the hashed directories that have backup.mbz files ... change into one and
head -n 40 moodle_backup.xml
That will show course name and ID. Go visit the course via Web.
Also what is destination for autobackups? Back info moodledata/filedir/ is default. I prefer those go to a designated directory manually created where I can count 'em and see 'em without having to mess with a DB query to see where they are in the maze or filedir.
If you have a designated directory ... is that an SSD device or a mount point to NFS or what? Could be a bottleneck.
No science to this issue ... just a lot of investigation ... scratching head ... trying something ...
'SoS', Ken
Hi
Did you get to the bottom of this? Only one of my users has 5 restores stuck in a queue and I can't see them anywhere when I go in as an administrator and I need to stop them and get them out of the queue,
Any ideas how I find them and how I stop them as I was not the instigator I'm just the admin, but I can't seem to find them anywhere.
Thank you
Heather