I am currently trying to create a powershell script where I compare two directories with many subfolders and files. Show
If a file exists in both locations then remove it in one of the folders. Hope you can help me. EDIT: sorry for the little edit but I thought to also mention the idea behind this. I am trying to do a backup with this powershell script which copies all files older 10 years to a destination and deletes it in the source directory. For sure the directory structure needs to be the same in the destination.
Traditionally, Windows users have employed XCOPY or SyncToy to mirror copy a certain directory to a different location. If you’re using Microsoft SyncToy, especially on Windows 10 or 11, it’s possible that sometimes the files or folders may be missing in the destination, or the destination folder has extra files or folders that are not in the source folder. Fortunately, nowadays, were have many specialized freeware sync tools that can compare and auto-sync your folders. This article tells you how to compare the directory structure (along with files) in two different directory paths to make sure you have an exact replica or mirror copy of the source. Some of the methods below also discuss how to synchronize items in two folders after comparing. There are 13 folder comparison methods (for Windows) discussed in this article. Choose the one which suits you best. If you need to check file integrity after synchronization, use one of the following hash checksum verification procedures listed. Compare two folders using Microsoft WinDiffWinDiff is a graphical file comparison tool from Microsoft which was first released in the year 1992 and subsequently updated. Please hold on! Don’t be taken aback by the year of the initial release. WinDiff is still awesome and it works perfectly fine in Windows 10, but no Unicode support. It has enormous potential. WinDiff can compare ASCII & Binary files, compare two directories, and synchronize the folders (left folder → right folder, or vice-versa). WinDiff download link (local mirror). The last updated WinDiff version is 5.2.3790.0 with the Date modified stamp showing 4-7-2016. You can also download WinDiff as part of the Windows XP Support Tools package, and extract its contents using 7-Zip. To do a directory comparison using WinDiff:
The results or the outline window by default shows identical files as well as the differences. You can hide identical files by disabling Show Identical Files from the Options menu. As you can see, WinDiff is not just comparing file names but also compares file contents in both locations and highlights changes if the file contents are different. Double-clicking on the “different” item opens the comparison view with color-coding. The color-coded results indicate what the file differences are. WinDiff can do ASCII and binary comparison. This means, WinDiff doesn’t rely upon modified date but compares the actual contents of the files. For example, I modified a character in a text file in the destination folder (Right folder). I replaced an exclamation mark with a colon, and the file size is exactly the same (1,127 bytes) in both locations. WinDiff picked it up and showed that the files are different. To save the directory comparison differences to a file, click Save File List in the File Menu. Type a file name with complete path without double-quotes. Uncheck Identical files and Include Checksums options, and click OK. The output file will look like this: -- D:\Websites\Winhelponline\blog\wp-content\uploads\2019 : E:\RameshPC-Synced Backup\Websites\Winhelponline\blog\wp-content\uploads\2019 -- includes left-only,right-only,differing files .\accesschk.exe only in E:\RameshPC-Synced Backup\Websites\Winhelponline\blog\wp-content\uploads\2019 .\compare-folders.txt only in E:\RameshPC-Synced Backup\Websites\Winhelponline\blog\wp-content\uploads\2019 .\fta-1803-w10.txt only in E:\RameshPC-Synced Backup\Websites\Winhelponline\blog\wp-content\uploads\2019 .\ssh-commands-useful.txt only in E:\RameshPC-Synced Backup\Websites\Winhelponline\blog\wp-content\uploads\2019 .\05\dell-supportassist.png only in D:\Websites\Winhelponline\blog\wp-content\uploads\2019 .\05\disable-theme-sync-ramesh.png different (E:\RameshPC-Synced Backup\Websites\Winhelponline\blog\wp-content\uploads\2019 is more recent) .\05\file-explorer-not-highlight-files.png only in D:\Websites\Winhelponline\blog\wp-content\uploads\2019 .\05\firefox-forget-about-this-site-4.png different (E:\RameshPC-Synced Backup\Websites\Winhelponline\blog\wp-content\uploads\2019 is more recent) .\05\msconfig-disable-services-non-microsoft.png only in D:\Websites\Winhelponline\blog\wp-content\uploads\2019 .\05\windows-store-error-0x800706d9.png only in D:\Websites\Winhelponline\blog\wp-content\uploads\2019 .\06\notepad-bing-search.png only in E:\RameshPC-Synced Backup\Websites\Winhelponline\blog\wp-content\uploads\2019 .\06\rundll32-refresh-commands.txt only in E:\RameshPC-Synced Backup\Websites\Winhelponline\blog\wp-content\uploads\2019 .\06\archives\404chk.vbs only in E:\RameshPC-Synced Backup\Websites\Winhelponline\blog\wp-content\uploads\2019 .\06\archives\actxprxy-missing-issues.txt only in E:\RameshPC-Synced Backup\Websites\Winhelponline\blog\wp-content\uploads\2019 .\06\archives\apps.png only in E:\RameshPC-Synced Backup\Websites\Winhelponline\blog\wp-content\uploads\2019 .\06\archives\articles.vbs only in E:\RameshPC-Synced Backup\Websites\Winhelponline\blog\wp-content\uploads\2019 .\06\archives\bash-sh.vbs only in E:\RameshPC-Synced Backup\Websites\Winhelponline\blog\wp-content\uploads\2019 -- 17 files listedSince the output file is in a tab-delimited format, you should be able to import into Microsoft Office Excel or Google Sheets easily.
Note that you can start WinDiff directly with source and destination paths as additional switches for directory comparison. For example, launching windiff.exe "c:\source" "d:\destination" would launch the tool and straight away compare those two folder paths. Here is the complete list of WinDiff command-line switches. Usage: windiff [options] path1 [path2] Options: -D Compare one directory only. -F[flags] savefile Save composite file to 'savefile'. The 'flags' may consist of one or more of I (identical), L (left), R (right), F (moved leFt), G (moved riGht), S (Similar left), A (similiAr right), X (exit after saving list). (e.g. -FLF saves list of Left or moved-leFt lines). -I file Reads list of files to compare, from the specified input file. Each line can contain one or two filenames, space delimited (with quoting, if filenames contain spaces). Use "-" as the filename to read from stdin. If a line contains only one filename, the file is compared to itself. -N name NET SEND notification to 'name' at end of comparison. -O Outline view (no automatic expansion). -P Perverse comparison: breaks lines on punctuation. -S[flags] savefile Save list of files to 'savefile'. The 'flags' may consist of one or more of S (same), L (left), R (right), D (different), X (exit after saving list). (e.g. -SLD saves list of Left or Different files). -T Compare whole subtree.To save the comparison report to a file using the -S switch. Examples are below: Save the list of identical files to diff.txt: -SS D:\output\diff.txt c:\folder1 d:\folder2Save the list of different files to diff.txt: -SD D:\output\diff.txt c:\folder1 d:\folder2Save the list of left-only files to diff.txt: -SL D:\output\diff.txt c:\folder1 d:\folder2Saves the list of right-only files to diff.txt: -SR D:\output\diff.txt c:\folder1 d:\folder2Saves the list of left-only AND right-only files to diff.txt: -SLR D:\output\diff.txt c:\folder1 d:\folder2To exit WinDiff automatically after saving the comparison report, add the X switch, as below: Save the list of right-only files to diff.txt and exit WinDiff -SRX D:\output\diff.txt c:\folder1 d:\folder2Save the list of left-only AND right-only files to diff.txt and exit WinDiff -SLRX D:\output\diff.txt c:\folder1 d:\folder2To generate all 5 individual “log” files you would have to run WinDiff 5 times. Example: windiff -SSX identical.txt c:\folder1 d:\folder2 windiff -SDX different.txt c:\folder1 d:\folder2 windiff -SLX left-only.txt c:\folder1 d:\folder2 windiff -SRX right-only.txt c:\folder1 d:\folder2 windiff -SLRX left-right-only.txt c:\folder1 d:\folder2See also WinDiff Help File Contents (windiff.hlp) in HTML format. Sync folders with WinDiffWhat’s more? WinDiff can also copy the missing or different files to the left (Folder 1) or to the right (Folder 2) location. To do so,
Despite the excellent ASCII & binary comparison and basic sync capabilities, WinDiff is often an underrated tool among IT folks. That’s because it sports an outdated user interface and has no Unicode support. But WinDiff is enough for most of us, especially those who use the English language OS. Microsoft has a knowledgebase article on WinDiff, titled How to Use the Windiff.exe Utility which you may want to check out. Here is how to compare two folders along with sub-folders, using WinMerge.
The advantage of using WinMerge is that it offers five different file compare methods to choose from:
WinMerge supports a lot of command-line switches. Check out the WinMerge Manual for the entire list of command-line switches supported. If you have unchecked the Show Identical Items from the View menu in WinMerge, it only shows you the list of modified, left-only and right-only files. It’s easy to synchronize the items from left → right, or vice versa. All you need to do is select the files (or select all), right-click on the selection, and click Copy. In the Copy sub-menu, choose one of the options:
Method 3: How to Compare Files in Two Folders Using FreeFileSyncFreeFileSync is an open Source software folder comparison and synchronization software that creates and manages backup copies of all your important files.
FreeFileSync offers the following synchronize options similar to Microsoft SyncToy.
By default, it does 2-way sync. Personally, I prefer the Mirror backup for my website projects and Update backup option for documents. Choose one of the synchronize options accordingly. In the comparison results, all the files are selected by default. You can exclude a file from being synchronized by unchecking the checkbox adjacent to the file name, or via the right-click menu.
That’s it! Your folders are now synchronized and the copy/update/delete operations are logged by FreeFileSync. Compare & Sync a Local folder Vs. FTP folder recursivelyFreeFileSync can also compare a local folder versus Google Drive, or a local folder versus a folder your FTP server recursively using FTP or SFTP connection. Here is how to compare and synchronize a local folder and subfolders with a folder on your FTP server. All you need to do is click on the Access online storage button next to the Browse button on the left pane or the right side. Clicking that button opens the following dialog: Input your FTP or SFTP connection information, credentials, and the FTP directory path to compare against your local folder. Once done, you’re now back at the Comparison window. Chose File size or File time and size comparison method. The latter comparison method is more reliable. Select the synchronization variant. I’ve set mine to 2-way sync (instead of Mirror) for this purpose. Once configured, click Compare. You’ll see the list of missing and different files on the left as well as the right folder. Click on the Synchronize button to copy files both ways, as 2-way sync has been selected. If you want to update only the FTP location, choose Mirror sync instead. You can save the above configuration to a batch job and run it via command-line or Scheduler. To save the configuration and run it using command-line, follow these steps:
Tip: FreeFileSync can also sync folders against Google Drive. Method 4: How to Compare Files in Two Folders Using SyncFoldersSyncFolders is another powerful file compare and synchronization utility that runs on all versions of Windows, including Windows 10. The program allows you to easily copy new and updated files and folders in either or both directions. You can also do a mirror backup (equivalent to Robocopy /MIR or SyncToy’s Echo feature.) From the SyncFolders dashboard, click on the Create rule.. button. Mention the source and target folders, and the type of action (backup method) you want to create.
Trivia: Other sync tools (e.g. FreeFileSync) use database comparison only for 2-way synchronization. But SyncFolders uses database comparison even for Left → Right mirror synchronization. The database file is named Syncfolders_Database_db stored in the source as well as the target location for each folder pair. Note: If you want to mirror backup the source folder, choose the Back up option. This option is equivalent to Robocopy’s mirror option or Microsoft SyncToy’s Echo feature or FreeFileSync’s Mirror option. Here are the configuration screens of the SyncFolders utility: In the above screenshot, you can see the option named Verify synchronization results by hashing (CRC32 or MD5 or SHA-1). If this option is enabled, after the copy/sync operation, SyncFolders will calculate the hash of the source file and the destination file and compares if the operation was successful. Hash verification is a resource-intensive task especially when large files are involved. Note that you can also compare files by content, in which case, the utility may be doing a binary compare (byte-by-byte comparison.) This can consume more CPU resources and needs to be used only when necessary. Compare folders using SyncFilesFrom the main window or from the processing tab, click on the Scan button to view the stats. It shows the number of files and folders to be synchronized, the total size, orphaned files and folders to delete (in the target location), and other details. To view the list of files and folders, open the compare window by clicking on the Details… button. In the compare window, you can see the list of new or modified items that are going to be copied. You’ll also see the extra files in the target folder that’ll be deleted during the synchronization operation. To change the “action” for a specific file or folder, right-click on it and choose the required option. For instance, you may want to do one of these things for an item:
Synchronize folders using SyncFilesYou can synchronize folders via the main window (Rules tab or the Processing tab), or from the compare window above. Clicking on the Synchronize button is all you need to do. The results are shown in the Processing tab, as well as logged in a file named SyncFolders.log. To view the log, click on the chevron near the gear button in the toolbar, and select View Log File. You’ll see something like this in the log file: <-------------------- PROCESS START --------------------- Process started at : 11/6/2020 9:14:42 AM Process stopped at : 11/6/2020 9:14:43 AM Rules File : D:\Users\ramesh\Desktop\SyncFoldersPortable\User Data\ramesh\My Rules.rls Executed rules : 001 - Website assets backup Files Synchronized : 12 Files deleted : 2 Folders deleted : 1 MB Synchronized : 0.15 MB Deleted : 0.02 Number of conflicts : 0 Number of Errors : 0 --------------------- PROCESS END ---------------------->Scheduled Task & Command-line usageYou can easily schedule the synchronization task using SyncFolders.
Here’s the command-line syntax it uses to synchronize folders (using a custom rule file) non-interactively using Task Scheduler. SyncFolders.exe "drive:\path\My Rules.rls" /background /synchronizeIf you don’t want to create a scheduled task, you may create a desktop shortcut for the above command to synchronize folders manually. SyncFolders is certainly an impressive piece of software! It is portable and has a neat and simple user interface. The filtering options in this tool are very useful. The compare window is the one that needs improvement, though. It’s nice for synchronizing tasks and can be a good alternative for the outdated Microsoft SyncToy. Method 5: How to Compare Files in Two Folders Using TotalCommanderTotal Commander is a file manager for Windows that lets you copy, move, or delete files. Total Commander can do much more than that. It can pack and unpack files, access FTP servers, compare files by content, etc.
To start “Synchronize dirs”, you can use the following command-line syntax: TOTALCMD64.EXE /S=S d:\folder_1 d:\folder_2Then, configure the options and press the Compare button. To start “Synchronize dirs” and compare folders right away, use this syntax: TOTALCMD64.EXE /S=S:= d:\folder_1 d:\folder_2(the comparison will start with the passed directories and last used options.) Optionally, you can also type in a settings name in the form /S=S:SettingsName so that the comparison will start immediately based on that previously saved setting.
Tip: Using Total Commander, you can even synchronize a local directory and a directory on an FTP server. Method 6: Compare & Synchronize folders in Real-time using DSynchronizeDSynchronize is a nice tool using which you can synchronize two folders manually or automatically in real-time. This tool provides a lot of options than any other folder sync tools for you to configure how files are synchronized.
To synchronize folders, uncheck the Preview sync option under the General section, and click Synchronize. It does a Mirror mode sync by default. Mirror mode has been discussed in this article earlier. Mirror mode ensures that the right folder is exactly the same as the left folder. Any extra directories and files in the destination are removed automatically. Real-time synchronizationTo synchronize two folders in real-time:
I copied some files to the left folder (source), and within 10 seconds, the items were synchronized to the right folder automatically. You can see that in this animated GIF: And, the changes (file or folder additions & deletions) are logged in the main window. When real-time synchronization is enabled, it monitors for file changes and the program sits in the notification area waiting for changes. It doesn’t take much CPU resources or memory. Last I checked, it used only around 10 MB of memory. Here is the list of salient features of DSynchronize:
Command-line supportYou can use the following command-line to automate DSynchronize: /START = Start synch with last saved options. The job name can be customized by clicking on the JOBS button at the top of the program’s main window. For example, to automatically sync two folders (previously saved job named “Uploads”), I’d run this command: "D:\dsynchronize\DSynchronize.exe" /STARTJOB "Uploads" /MINIMIZETo synchronize the two folders (job named “Uploads) and activate real-time sync, run: "D:\dsynchronize\DSynchronize.exe" /STARTJOB "Uploads" /AUTOREALTIMEThe above command synchronizes the two folders named in the job and activates real-time monitoring automatically. You can see the real-time monitoring icon in the Notification area. That’s it! HashMyFiles is a small utility that calculates hash checksums for your files. It can compute the hash for files in a folder and its subfolders recursively. HashMyFiles lets you save the results to a file. It can also compute hash checksum via the right-click menu. If PowerShell is not your cup of tea, use the HashMyFiles utility from Nirsoft.net to compare file hashes. You can save the file names and the corresponding hash output for the source folder tree to source.txt. Similarly, save the entries for the destination folder tree as destination.txt. In this example, we’re going to compare files in the following folders: D:\Websites\Winhelponline\blog\wp-content\uploads\2019 E:\RameshPC-Synced Backup\Websites\Winhelponline\blog\wp-content\uploads\2019Follow these steps to generate file hashes using HashMyFiles and compare them:
You’ll see the list of different files, extra files, and missing files. Let’s now see how to use the built-in Robocopy console utility to compare two folders. Robocopy (“Robust File Copy for Windows”) is an excellent file copy tool built-in to Windows that has enormous capabilities. Robocopy’s capabilities are above and beyond the built-in Windows copy and XCopy commands. RoboCopy is the most preferable of all the command-line methods because Robocopy compares not only the file names but also the last modified time. It reports the list of modified files and new files/folders in the source and destination paths. Among its excellent features include “mirror” mode copy (similar to Microsoft SyncToy’s “Echo” option), ability to retry copying in case of network interruptions, multi-threaded copying, etc. Other than copying files, we can use Robocopy to just compare the contents of two folders (recursively) and log the differences without copying anything, like a dry run. ScenarioMy website’s files are synced to an external hard drive (E:\) regularly. For the purpose of illustration, let’s check if these two folders are identical. In this example, we’re going to compare the following folders, recursively (i.e. along with sub-folders, a deep comparison): D:\Websites\Winhelponline\blog\wp-content\uploads and E:\RameshPC-Synced Backup\Websites\Winhelponline\blog\wp-content\uploads
Details about the Robocopy command-line switches used
In this example, we’re writing the output to D:\Diff.txt. Here is what the log looks like: ------------------------------------------------------------------------------- ROBOCOPY :: Robust File Copy for Windows ------------------------------------------------------------------------------- Started : Monday, April 8, 2019 5:06:10 PM Source : D:\Websites\Winhelponline\blog\wp-content\uploads\ Dest : E:\RameshPC-Synced Backup\Websites\Winhelponline\blog\wp-content\uploads\ Files : *.* Options : *.* /TS /L /S /E /DCOPY:DA /COPY:DAT /PURGE /MIR /R:1000000 /W:30 ------------------------------------------------------------------------------ 583 D:\Websites\Winhelponline\blog\wp-content\uploads\ *EXTRA Dir -1 E:\RameshPC-Synced Backup\Websites\Winhelponline\blog\wp-content\uploads\2019\videos\ *EXTRA File 821886 2019/04/07 07:19:24 tiktok-superman-dance.mp4 3 D:\Websites\Winhelponline\blog\wp-content\uploads\2019\Scripts\ Older 210 2019/04/08 10:54:59 0x80071A91.vbs Newer 211 2019/04/08 11:35:32 ErrCodeFinder.vbs New File 211 2019/04/08 11:35:32 Sample.vbs 211 D:\Websites\Winhelponline\blog\wp-content\uploads\h1-10\ 83 D:\Websites\Winhelponline\blog\wp-content\uploads\h1-11\ 1 D:\Websites\Winhelponline\blog\wp-content\uploads\reg\ 9 D:\Websites\Winhelponline\blog\wp-content\uploads\temp\ ------------------------------------------------------------------------------ Total Copied Skipped Mismatch FAILED Extras Dirs : 113 0 113 0 0 1 Files : 4125 3 4122 0 0 1 Bytes : 59.96 m 632 59.95 m 0 0 802.6 k Times : 0:00:00 0:00:00 0:00:00 0:00:00 Ended : Monday, April 8, 2019 5:06:10 PMYou can clearly see the extra directories and files in the source as well as the destination folders. Also, it compares the timestamp and tells you if files are older or newer (compared to the same file in your destination). In this example, here are the differences spotted:
Complete list of RoboCopy file classifications: ----------- ----------- ---------------- --------------- ------------- ------------ Classifi_ Exists In Exists In Source/Dest Source/Dest Source/Dest cation Source Destination File Times File Sizes Attributes ----------- ----------- ---------------- --------------- ------------- ------------ Newer Yes Yes Src > dest n/a n/a Older Yes Yes Src < dest n/a n/a New Yes No n/a n/a n/a Extra No Yes n/a n/a n/a Changed Yes Yes Equal Different n/a Same Yes Yes Equal Equal Equal Tweaked Yes Yes Equal Equal Different Mismatched Yes (file) Yes (directory) n/a n/a n/a ----------- ----------- ---------------- --------------- ------------- ------------Note: A Tweaked file is defined to be one that exists in both the source and destination, with identical size and timestamp, but different attribute settings. Robocopy is an amazing tool to compare & sync files in two folders (along with sub-folders several levels deep) and list out the differences in a log file. To synchronize folders (only from the left → right folder) with RoboCopy, use the same command-line syntax we used earlier to compare folders, but this time without the /L (list-only) switch.
RoboCopy’s Mirror copy mode (/MIR) is similar to Microsoft SyncToy‘s “Echo” feature. Remember, RoboCopy does only left → right synchronization — NOT a two-way synchronization. It’s important to note that the /MIR switch also removes any extra files and directories in the destination folder tree. The “Extra” files/folders are items present in the destination but not present in the source path. The /MIR (mirror) command makes sure that the destination folder structure (and files) is an exact copy of the source. Use the following command-line syntax to sync items from left → right folder: robocopy source_folder destination_folder /MIR /MTThe /MT switch is to do a multi-threaded copy for a super-fast copy operation. It’s an optional switch, but extremely useful. The sync status is outputted in the console for each item processed, and the statistics are shown at the end of the output. To log the output to a file, use this syntax: robocopy source_folder destination_folder /MIR /MT /TS /LOG:D:\Diff.txtAutomate the commands using a Batch fileYou can add all the Robocopy commands to Notepad, and save it as a Windows Batch file (with .bat extension). Simply double-click on the Batch file whenever you want to sync the mentioned folders. You may even run the batch file at the required intervals using Task Scheduler. Method 9: Compare Two Folders using File Hash Checksum with the FCIV utilityUsing the cryptographic hashing algorithm — e.g., MD5, SHA-1, you can verify if two files are identical or not. The checksum is a hash value used for performing data integrity checks on files. It’s a kind of signature for a file. The Microsoft File Checksum Integrity Verifier tool (FCIV) is a command-line utility that computes MD5 or SHA1 cryptographic hashes for files. You can output the list of files along with their MD5 or SHA1 hashes to an XML database, and then compare files in the target folder with the hashes stored in the XML database file. This method is especially helpful after you’ve used one of the above methods — e.g., Robocopy or WinDiff to synchronize two folders, and want to verify the integrity of each destination file by comparing the file hash/checksums with that of the source location. Scenario: Let’s compare the contents of the following folders recursively using FCIV: D:\Websites\Winhelponline\blog\wp-content\uploads E:\RameshPC-Synced Backup\Websites\Winhelponline\blog\wp-content\uploadsDownload Microsoft File Checksum Integrity Verifier from Official Microsoft Download Center, and extract fciv.exe to a folder — e.g., d:\tools. Then follow one of these methods: Option 1
The above method is very helpful to check the integrity of matching files in the destination folder. But it doesn’t show the list of extra files in the destination. That’s where Option 2 below has an additional advantage. Option 2 Rather than outputting the files list and checksums to an XML database, you can output the items in the source and destination to separate text files. Then the text files can be compared using WinDiff, WinMerge, DiffChecker.com or WinCompare utility. Run these commands from a Command Prompt window. fciv -r "D:\Websites\Winhelponline\blog\wp-content\uploads" -bp "D:\Websites\Winhelponline\blog\wp-content" >d:\source.txt fciv -r "e:\RameshPC-Synced Backup\Websites\Winhelponline\blog\wp-content\uploads" -bp "e:\RameshPC-Synced Backup\Websites\Winhelponline\blog\wp-content" >d:\destination.txtThen, compare the two files source.txt and destination.txt using a file compare tool like WinDiff, WinMerge, DiffChecker.com, DiffMerge or WinCompare. You’ll see the list of missing files, extra files and modified files (based on the computed checksum) in the two folder locations. I synchronized the two folders using RoboCopy and then compared the folders again using FCIV hash checksum. They’re now identical! Method 10: Compare Two Folders Using PowerShellUsing PowerShell you can recursively compare two folders easily. Start PowerShell. Copy the following snippet and paste them all in the PowerShell window, and press ENTER Enter the source folder and destination folders, replacing the source_folder_path and dest_folder_path placeholders respectively. $Folder1 = "source_folder_path" $Folder2 = "dest_folder_path" function Get-Directories ($path) { $PathLength = $path.length Get-ChildItem $path -Recurse | % { Add-Member -InputObject $_ -MemberType NoteProperty -Name RelativePath -Value $_.FullName.substring($PathLength+1) $_ } } Compare-Object (Get-Directories $Folder1) (Get-Directories $Folder2) -Property RelativePath | Sort RelativePath, Name -descYou’ll see an output like this: The side indicator <= means the file or folder exists only in the source. i.e., missing in the destination. Editor’s note: The above code compares only the file names without matching any other details like size, date modified, etc. PowerShell has the capability to do a hash-comparison of items in two folders and output the results, which is described in the next method. Sync folders With some 10 additional lines of PowerShell code, you can even copy the different or missing files from source → destination, or vice versa. For more information, see Sync Files And Folders With PowerShell. To compare files in two locations using file hash checksum, you can use PowerShell. PowerShell by default uses the SHA-256 algorithm, which you can change using the -Algorithm parameter. As said earlier, hash checksum is a kind of signature for a file which is unique. In this example, we’re going to compare the following folders: D:\Websites\Winhelponline\blog\wp-content\uploads\2019 E:\RameshPC-Synced Backup\Websites\Winhelponline\blog\wp-content\uploads\2019Use the following command-line to compute the hash for files in the source folder and output the results to source.txt file. Get-ChildItem 'D:\Websites\Winhelponline\blog\wp-content\uploads\2019' -Recurse | Get-FileHash | Select Path, Hash | export-csv d:\source.txt -Delimiter "`t" -NoTypeInformationThen, run the following command to remove the base paths from the file. (gc -path d:\source.txt -raw) -replace 'D:\\Websites\\Winhelponline\\blog\\wp-content\\', '' | Out-File d:\source.txtThe list of file names (with relative path) along with the SHA-256 hashes are outputted to the source.txt file. Similarly, run the commands for computing hashes for items in the destination, and then remove the base paths from destinations.txt: Get-ChildItem 'E:\RameshPC-Synced Backup\Websites\Winhelponline\blog\wp-content\uploads\2019' -Recurse | Get-FileHash | Select Path, Hash | export-csv d:\destination.txt -Delimiter "`t" -NoTypeInformation (gc -path d:\destination.txt -raw) -replace 'E:\\RameshPC-Synced Backup\\Websites\\Winhelponline\\blog\\wp-content\\', '' | Out-File d:\destination.txtThe above command computes the hash for each file in the destination folder recursively, and the output is written to d:\destination.txt Compare using PowerShell Next, compare the files using the following PowerShell command: Compare-Object -ReferenceObject $(Get-Content d:\source.txt) -DifferenceObject $(Get-Content d:\destination.txt) | flIt shows the different files and the location of each item denoted by the symbols <= or => for left folder and right folder respectively. The output also shows extra files in the source as well as the destination. Compare using WinDiff Alternately, you can run WinDiff and compare d:\source.txt and d:\destination.txt. You can use the following WinDiff command-line to compare the files: d:\Tools\windiff.exe d:\source.txt d:\destination.txtThe WinDiff output shows differences highlighted in yellow/red. From the above screenshot, we can infer that the file 0xc1900101-error.png is different. Also, the destination folder or the right folder (denoted by the symbols !> in WinDiff) has an extra file named 1h.txt. PowerShell supports many hash algorithms and so it has an advantage over the FCIV.exe tool. The Microsoft FCIV tool supports MD5 and SHA-1 algorithms only, which are outdated and can be ineffective due to collision issues, albeit very rarely. SHA-1 collision means that two files have identical SHA-1 hashes but different content. So, SHA-256 may be the preferred algorithm for file or folder comparison in a production environment. Note that the command-line methods 11 & 12 below only help you compare the structure of two directories along with files. They don’t compare file sizes or modified date to know which is the newer. They only do a plain comparison based on file names and you wouldn’t know if one of the destination files becomes corrupt during a copy operation. In a production environment, you may need to use other tools or methods if you need to verify the integrity of your files in the source and destination folders, to make sure they are the exact same copies.Method 12: List the folder structure using TREE commandTo compare two directories along with sub-directories, the good old TREE command comes extremely handy. Suppose you want to compare the two folders (including file names) D:\SOURCE and D:\DEST, follow these steps:
Method 13: List folder structure using the DIR CommandTo compare the contents of two directories, another command-line you can use is the DIR command. Use the following command-line syntax to generate the list of files in a directory recursively. You can then compare it using a compare tool or DiffMerge. DIR D:\SOURCE /S /AD /B >D:\SOURCE.TXT DIR D:\DEST /S /AD /B >D:\DEST.TXTQuick Tip: To include files along with the directory listing, remove the “/AD” flag. There is one problem though, when using the DIR command. It lists full directory and file paths in the output, as below:
In that case, you’ll need to remove the comparison base paths D:\SOURCE\ and D:\DEST\ in the respective text files by editing them with Notepad. To bulk replace text, use the Replace option in Notepad and remove every occurrence of the base path. Once done, save the files and compare the contents online via DiffChecker or using an offline text-comparison tool like WinDiff or WinMerge. Directory comparison – List generated using DIR instead of TREEI bet you’ll find directory contents output generated using TREE to be much easier and far neater than DIR. Hope the above methods helped you quickly compare two folders recursively, identify missing and modified files, and reconcile differences. Let’s know your comments! |