I think your filescleanupperiod setting could be the problem that your trashcan stays that big as you describe.
I see this is also a new setting that wasn't present in Moodle 3.5. I find it strange that during your automated backup process the trashcan is growing, only files that are deleted are backups inside your .mbz directory and not files in your moodledata folder, or am I wrong? I thought the backup process only deletes the older backups and the temporary course in the backup folder. But these files aren't part of the moodledata folder.
I run the automated process outside Moodle cron and with the Linux cron that runs a separate script I wrote for the automated backup. This script runs every day and excludes the course that runs 10 hours to make a backup.
Here you can find my post about this subject, because of NFS we used I ran the automated backup process on the DB server, it was much faster then. Now we don't use NFS anymore, the automated backup runs on the Moodle server again:
https://moodle.org/mod/forum/discuss.php?d=383815
I included the current shellscript that runs on our Moodle server, I changed the automated backup_auto_active in 'manual', to run the process outside Moodle cron.