Currently I am working for one of our clients which offers a Self-Service vRealize Automation portal and is active as a Service Provider. All Backup and Restore activities are controlled in their Veeam environment. Whilst this is quite a huge environment with a lot of tenants, so there are hundreds of backups and restores taking place. The tenant can specifically choose to add his/her Virtual Machines from within the vRealize Automation Portal to a backup job, and as you can imagine he/she can also choose to delete this particular Virtual Machine from a backup job or disk. This can include all RestorePoints of this Virtual Machine.
While I recently upgraded the Veeam test environment to Veeam B&R 9.5 update 3 (build: 9.5.0.1536) we stumbled upon some errors if a tenant chose to delete his Virtual Machine(s) including the RestorePoints belonging to that Virtual Machine from disk. It is important to know that through the Veeam API (Veeam Management Server) it is not possible to delete backupsjobs and restorepoints. So we do this through PowerShell.
So the portal came back with the failed task, we tried to find the specifics of this failure. After isolating the problem we did a manual execution of the script. This is was happened:
But if we tried the exact same code on newly created jobs we did not get the error and it did what it had to do, and that would be : delete the specific restorepoints.
The first thing that crossed my mind was it actually looks like an issue after the Veeam upgrade to update 3. But to be sure I needed to conduct some more research.
First, we search for tenant specific jobs:
This represents all the jobs found for the YXC tenant, and as you can see, it finds a job with 2 restorepoints (what we want). Regarding this scenario Veeam asks for removal confirmation with such message performing the operation "Remove-VBRRestorePoint" on target "target_name". [Y] Yes [A] Yes to All [N] No [L] No to All [S] Suspend [?] Help (default is "Y"):Y
We wait for the confirm message and then de error is displayed.
Here is the example of the script:
So, we could not troubleshoot any further, it was time to log a case with Veeam Support. After uploading the database and configuration we finally got confirmation that this behavior is an actual issue in Veeam 9.5 u3, and that the Veeam R&D team is going to create a private fix.
Basically, the error occurs if there are ":" symbols in a job name, only PowerShell is affected.
After receiving the hotfix the problem was solved.
If you run in this kind of behavior and have BackupJobs that have “:” symbols in the job name, please log a case and ask if you could get the hotfix for this problem.
Arjan Bodde