How To Run RoboCopy Backup in Parallel

From time to time Windows Admins will surprise you with band-aid and bubble gum scripts, that’s entirely expected I think. But then again with just about the same irregular interval with a touch of entropy sprinkled on the electrical-fire cake, someone comes along and actually puts together something that works well, and efficiently. Resource efficiency is entirely under-rated and feels like the subject has been coaxed out of conversational existence by the x86 Wintel side of people, for lack of a better term.

In my opinion in order to achieve an efficient, fast executing, and fast to exit backup routine means executing multiple copies of Robocopy running in parallel through the use of spawning multiple copies via start.exe. The reason  for my preference of Robocopy, over say a custom PowerShell function is due to the fact that utilities like robocopy.exe is already in the most “efficient” form as seen by the physical CPU. Compiled binary executables (exe’s) will always win versus managed-code which is what PowerShell and regular batch scripts are. Just remember that the lower-level the code (C/C++, Assembly), the closer it is to native machine language, and execution faster due to less compiling needed for translation in binary (0’s and 1’s), in general. From the processor perspective these are apple’s and oranges. Without further adieu:

robocopy-loop.bat

@ECHO OFF
cd /d %~dp0

::: Generate reproducable new line variable
set nl=^&echo.

for /F "tokens=1-4 delims=/-. " %%i in ('date /t') do (call :set_date %%i %%j %%k %%l)
goto :end_set_date

:set_date
if "%1:~0,1%" gtr "9" shift
for /F "skip=1 tokens=2-4 delims=(-)" %%m in ('echo,^|date') do (set %%m=%1&set %%n=%2&set %%o=%3)
goto :eof
:end_set_date
::: End set date

for /F "eol=; tokens=1,2* delims=," %%i in (robocopy-paths.txt) do (call :process %%i %%j %%k)
goto :eof
:process
start /B robocopy %1 %2 %3 %4 %5 %6 %7 %8 %9
::: More error handling: || echo FAILED: %0 %1 %2 %3 %4 %5 %6 %7 %8 %9
set exit_code=%ERRORLEVEL%
IF ERRORLEVEL 1 (  
    echo FAILED: %1 %2 %3 %4 %5 %6 %7 %8 %9, return code = %exit_code%
    exit /b %exit_code%
)
exit /b %exit_code%
goto :eof

robocopy-paths.txt

%userprofile%\Documents, \\domain.com\Public\DFS\visualblind\Documents, /MIR, /ZB, /XO, /XD "C:\Users\visualblind\Documents\Sites", /NP, /NFL, /LOG+:%userprofile%\Documents\Temp\robocopy_%mm%_%dd%_%yy%.txt
%userprofile%\Desktop, \\domain.com\Public\DFS\visualblind\Desktop, /MIR, /ZB, /R:2, /XO, /NP, /NFL, /LOG+:%userprofile%\Documents\Temp\robocopy_%mm%_%dd%_%yy%.txt
%userprofile%\Downloads, \\domain.com\Public\DFS\visualblind\Downloads, /MIR, /ZB, /R:2, /XO, /NP, /NFL, /LOG+:%userprofile%\Documents\Temp\robocopy_%mm%_%dd%_%yy%.txt
%userprofile%\Pictures, \\domain.com\Public\DFS\visualblind\Pictures, /MIR, /ZB, /R:2, /XO, /NP, /NFL, /LOG+:%userprofile%\Documents\Temp\robocopy_%mm%_%dd%_%yy%.txt
%userprofile%\AppData\Local\Game\Save, \\domain.com\Public\DFS\visualblind\AppData\Local\Game\Save, /MIR, /ZB, /R:2, /XO, /NP, /NFL, /LOG+:%userprofile%\Documents\Temp\robocopy_%mm%_%dd%_%yy%.txt
;echo Robocopy Done%nl%"%username%\Desktop, %username%\Documents, %username%\Downloads, %username%\Pictures, %username%\AppData\Local\Game\Save > \\domain.com\Public\DFS\%username%" >> %userprofile%\Documents\Temp\robocopy_%mm%_%dd%_%yy%.txt
%userprofile%\Documents, D:\%username%\Documents, /MIR, /ZB, /XO, /XD "C:\Users\visualblind\Documents\Sites", /NP, /NFL, /LOG+:%userprofile%\Documents\Temp\robocopy_%mm%_%dd%_%yy%.txt
%userprofile%\Desktop, D:\%username%\Desktop, /MIR, /ZB, /R:2, /W:55, /XO, /NP, /NFL, /LOG+:%userprofile%\Documents\Temp\robocopy_%mm%_%dd%_%yy%.txt
%userprofile%\Downloads, D:\%username%\Downloads, /MIR, /ZB, /R:2, /W:55, /XO, /NP, /NFL, /LOG+:%userprofile%\Documents\Temp\robocopy_%mm%_%dd%_%yy%.txt
%userprofile%\Pictures, D:\%username%\Pictures, /MIR, /ZB, /R:2, /W:55, /XO, /NP, /NFL, /LOG+:%userprofile%\Documents\Temp\robocopy_%mm%_%dd%_%yy%.txt
%userprofile%\AppData\Local\Game\Save, D:\%username%\AppData\Local\Game\Save, /MIR, /ZB, /R:2, /XO, /NP, /NFL, /LOG+:%userprofile%\Documents\Temp\robocopy_%mm%_%dd%_%yy%.txt
;echo Robocopy Done%nl%"%username%\Desktop, %username%\Documents, %username%\Downloads, %username%\Pictures, %username%\AppData\Local\Game\Save> D:\%username%" >> %userprofile%\Documents\Temp\robocopy_%mm%_%dd%_%yy%.txt

It is incredibly obvious, and stupidly simple to see the vast differences in resource consumption and vast resource waste coming from the Windows Server arena. Since the NT kernel is closed-source, not you or I can tell what the hell is going on in there. Due to the explosion of virtualization, and more specifically, guest VM resource report-ability, you can look at two idling VM’s running on the same physical hardware with the same specs (1 vCPU, 1GB vMem, etc.) and more often than not the Windows Server 2008 R2, 2012, 2012 R2, 2016, 2019 Standard/Enterprise/Datacenter Edition guest is consuming (hogging) at least 2x CPU cycles and consuming 2-3x vMem. Albeit we must also take into account that the Wintel VM is probably providing more numerous services per individual guest (CIFS, ADDS, DHCP, DNS, etc.), so one of these days if I have the time I’ll try launching a head-to-head Torvalds vs. Gates competition. Zero-MegaHertz sounds like a good name to me.