Recycle bin settings ... always show in course and time to retain to 1 second.
That should hurry along the process.
Also truncate the contents of mdl_task_adhoc.
That's a 'tracker' table - contains a nextruntime column.
In a site that has no issues, that table contains nothing.
Also in moodledata/temp/backup/ look for contenthashed named directories that have a date close to when you tried the async backups. Those might not get cleaned up by your moodle's cron job/task to clean up - so remove them manually.
Then try your command line backup of the course - and suggest sending that to a directory outside of moodledata/filedir/
If this is a very large course, might have to do a nohup [php command] & to put that into a no hang up (run forever) until completed. Issue that in code/admin/cli/. That does create a nohup.out log in the directory where the command was issued and you can watch that log to see when it completes.
'SoS', Ken