Hello.
I tried Ayoub Kazar's test files again, using Nazir Bilal Yavuz's v3 patches, but with one difference since my last attempt: this time, I used 5 million lines per file. For each 5 million line file, I ran the import 5 times and averaged the results.
(I found that even using 1 million lines could sometimes produce surprising speedups where the newer algorithm should be at least a tiny bit slower than the non-simd version.)
The text file with no special characters is 30% faster. The CSV file with no special characters is 39% faster. The text file with roughly 1/3rd special characters is 0.5% slower. The CSV file with roughly 1/3rd special characters is 2.7% slower.
I also tried files that alternated lines with no special characters and lines with 1/3rd special characters, thinking I could force the algorithm to continually check whether or not it should use simd and therefore force more overhead in the try-simd/don't-try-simd housekeeping code. The text file was still 50% faster. The CSV file was still 13% faster.