Have Geek Brain

How does the large text file viewer work? How to build a large text reader

how to work with a large text file ?

I suppose that:

  • Threading is used to process a file
  • TextBox updates line by line
  • Efficient memory handling is used

Are these assumptions correct? if someone had to develop their own, what is needed and not needed?

I want to implement one using DataGrid instead of TextBox

I like C ++ and python. I will probably be using QT / PyQT

EDIT

Files, I usually have from 1.5 to 2 GB. I am looking at editing and viewing these files.

+2


a source to share


2 answers


I believe the trick is not loading the entire file into memory, but using search and so on to just load the part being viewed (perhaps with a before and after block to handle a bit of scrolling). Perhaps even using memory mapped buffers, although I have no experience with them.



Understand that modifying a large (fast) file is different from just browsing. You may have to copy gigabytes of data surrounding your edit to a new file, which can be slow.

+6


a source


In Kernighan and Plaugher's classic (antique) book, Software in Pascal, they cover the development and design options of the ed (1) version and the notes

"Warning: edit

- a large program (excluding contributions from translit

, find

and change

; at 950 lines, that's 50% more than anything else in this book."



And they (literally) didn't even use string types. Because they note that the file to edit might exist on a tape that does not support arbitrary writes in the middle, they needed to store the position index in memory and work with the scratched file to store changes, deletions, and additions, putting everything together on command. " save ". They were, like you, concerned that memory limits the size of their editable file.

The general structure of this approach is preserved in the GNU ed project , especially inbuffer.c

+4


a source






More articles:


All Articles