Comparing large files using prediffer unsuitable

[OpenShooter]

I had an occasion recently to compare hundreds of large (~0.5 GB) text files between two directories. These files needed to be compared ignoring part of each line, columns 80 to the end of the line. It seems WinMerge has such a capability, but it operates by copying the file (prediffer), removing the parts to be ignored during comparison. This is unsuitable because it will be very slow, and it will require a lot of temporary extra disk space.

Is there any better way to make this comparison with the existing WinMerge 2.16.12.0?

Another question that came up during this comparison is excluding files from the comparison. The directories being compared actually contain 6 sets of files, Com kIx=nnn.txt, ComIxSeq kIx=nnn.txt, ComNoSeq kIx=nnn.txt, Pos kIx=nnn.txt, PosIxSeq kIx=nnn.txt and PosNoSeq kIx=nnn.txt. In this case, I only wanted to compare the "Pos kIx" set, but if I set Folder; Filter to "Pos*" and start the compare, the filename window is empty. Perhaps it just takes a few minutes to determine which files; if so, there needs to be some user feedback, e g, "Locating files...".

[sdottaka]

Do you mean the IgnoreColumns.dll prediffer plugin? Unfortunately, WinMerge is an implementation that makes heavy use of temporary files, so it's not very suitable for handling large files.

but if I set Folder; Filter to "Pos*" and start the compare, the filename window is empty

I tried the following, but couldn't reproduce the problem.

If it's taking a long time, you should see a progress bar.

[OpenShooter]

I do see this progress bar, but only after the compare results start appearing. Before that, I just see an empty window. Also, filter does not always work. For example, filter=Pos* includes filenames beginning with C. But now I notice it shows the C files as "File skipped", which is correct (but unwanted). I have ~4000 files in the directory, of which about 1/6 I want to compare, and I don't want the other ~3333 cluttering up my output. And the skipped file list is incomplete, seemingly at random. For instance, my directory has 462 C files, which are to be skipped per the filter, and, files 1-139 do not appear in the File skipped list, but files 140-462 do,

Another anomaly I have just noticed is that WinMerge seems to remember parts of a comparison between runs, and starts where it left off. I am not sure of the details of this behavior, though.

[sdottaka]

I do see this progress bar, but only after the compare results start appearing. Before that, I just see an empty window

I couldn't reproduce this issue.
Are you comparing any special folders?
Or are you running on a machine with a very large number of CPU cores?
If it is the latter, could you try setting the number of comparison threads to 1 as shown below?

For example, filter=Pos* includes filenames beginning with C. But now I notice it shows the C files as "File skipped", which is correct (but unwanted).

I may not understand the problem exactly, but if you want to compare both filenames starting with C and files starting with Pos, specify Pos*;C* for Filter. If you need more complex conditions, you need to create a filter file. See https://manual.winmerge.org/en/Filters.html#Filters_FileFilter for filter files.

If you have a problem that feels like a bug, please tell us how to reproduce it.

Another anomaly I have just noticed is that WinMerge seems to remember parts of a comparison between runs, and starts where it left off. I am not sure of the details of this behavior, though.

This problem could not be reproduced either.
If you know how to reproduce it, please let us know.