autobackups doesn't have any fine tuning to it ... think it uses course id regardless of the size of the course.
Suggest looking in moodledata/temp/backup/ for long directory names and inside there see if there is a backup.mbz file.
If there is, it means it failed on the last step of backup, that of copying ... notice that's copy ... to destintation - which by default is in the sea of files of moodledata/filedir/. Means that for a little while, drive space requirement is 2X the size of that mbz file.
How much free space do you have?
Command line backup works if giving a path to save the backup file outside of moodledata/filedir/. Web service taken out of the loop ... just php script talking to DB resulting in creation of mbz at given location.
BTW, any moodledata/temp/backup/longname subdirectory you see at that location ... check date/time stamps on file ... if old and autobackup not running in adhoc mode, those can be manually removed. Those are failed backup/restores and manually removing those directories can free up a lot of space!
Had a site once that attempted to use autobackups. That would fail when hitting a Electronic Media Course (digital photos, videos, and audios) whose total size was 130 gig - 25% of the space. In course ID's that course came early. Had to turn off auto and create 3 separate scripts for command line backup .. 2 for small to medium size courses and just one more for large courses .. the large course script ran only on weekends.
'SoS', Ken