Delete Duplicate Lines

Hello, I can create a multithreaded software for you but not for GPU. If you are interested in my offer please contact me so we could discuss details. Respectfully, Grig

Hi sir How do u think about .NET? C# support List (Enumerable ) class that could use distinct function. And for multi or big file no problem , i thnk thanks.

I shall be able to complete the project for given text file in 5 days. Please specify the type of file and type of text(ascii, binary,...) and is ther any requirement for delimiters for line ending ...should whitespaces be ignored...should it be plateform independent?

Dear Sir, Greetings from Velrajan!! I am from chennai.. I can do the job for you ( without CPU part) I will provide GUI, you can load multiple big files..i will make the program multithreaded... Looking forward your response.. Velrajan T.

I can deliver this Windows application for you which will allow deleting duplicate lines from multiple TXT files. It will work as fast as your CPU is. Can deliver in 3 days or less; Will provide support to get you started. Bests, Larry

A proposal has not yet been provided

I would first like to know why you need such a program? Any text editor can do this job efficiently. If you could send me more description that would be great.

Greetings for the Day!!! I have 6 years of experience in .NET, VBA Macros, VB script,PS Script and VB creation with application like SAP, Internet explorer, Microsoft Outlook,PDF& Text files, MS Access and SQL Server databases. And also I have worked on extraction with websites like Amazon, Cellpex,Costco,etc., hope if awarded with this project I can make it best and better with maximum 100% accuracy and satisfaction. please award me this project and contact me for further details Thanks Prabakar M

I'm sure to complete your job. Inbox me if you want to have more detail about this tool. Thank for your attention. HuyPQ.

We have very big experience with various PC applications - parsers, readers, configuration tools and etc. This task we can do with very high quality.

I can create app this app. If u want I can implement and search engine.

I would use c++ to complete this task. Please, let me know if I can use visual studio c++ 2010 (or similar) for it (the easiest way to me), because it is faster to create a file dialog to select the files. I have about 20 years programming in c/c++, and I can manage the standard c++ library very well to remove duplicate lines.

if the resulting file is smaller than 2gb then it will be handled in memory and will be fast enough. once it pass over that limit the execution time will decrease. if you are ok to run 64bit version the limit is higher.

Instead of GUI code, I can provide you a script that will do this superfast. I will use sed utility that is actually used in Linux for text related functions on file or strings. The utility is also available in windows.

My proposal involves the use of mapping files. A mapping file is a small (in comparison to the original) file that gives a number to each line of a file. This number corresponds to the characters in that line, if two numbers are similar, the lines are identical. Using a mapping file makes working with large file sizes possible. Project Outline: 1. A mapping file is created for each file to be compared. 2. Each line of mapping files are compared. 3. If lines are similar in multiple mapping files, that line gets flag. 4. A final file is made by combining all comparison files, while removing any duplicate files flagged in previous step.