[Cialug] Efficiently removing the beginning of a file

[David Bierce]

If your running a command like:

>> dd if=inputfile of=outputfile2 ibs=3635 obs=1M skip=

Depending on the media it is generally more efficient to use iseek  
instead of skip because it doesn't actually read the blocks.  It is  
generally safe, unless you're using something like tape.


Dave
On May 21, 2007, at 12:10 PM, Jeffrey C. Ollie wrote:

>> I have a ~70 MB file. The first 3635 bytes need to be removed.  
>> What is
>> the most efficient way to do that? I did this, knowing it would work
>> but would be slow:
>>     $ dd if=inputfile of=outputfile ibs=1 obs=1M skip=3635
>> It did indeed work. But it took 274 seconds (and pegged the CPU the
>> entire time), whereas simply copying the file with cp only takes 2
>> seconds. Since what i want to do is not *that* different an action  
>> from
>> just copying the file (at least in terms of the minimum disk  
>> operations
>> that would be required), it seems to me that there should be a way to
>> do it that only takes ~2 seconds. What are some other command line  
>> ways
>> to do this that would be more efficient?
>>
>> Actually, before hitting send i tried another test, just flipping the
>> "ibs" and "skip" values:
>>     $ dd if=inputfile of=outputfile2 ibs=3635 obs=1M skip=1
>> That only took 2.5 seconds, which is much closer to the theoretical 2
>> second time that should be possible. But i guess what i'm curious  
>> about
>> is the general problem; if there is a large file and you need to  
>> remove
>> some small number of bytes from the beginning of it, how is that best
>> accomplished? If i had needed to remove only 1 byte for example, i
>> would have had to have used "ibs=1 skip=1" which would have taken
>> around 274 seconds again.

-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://cialug.org/pipermail/cialug/attachments/20070521/21f16d5d/attachment.html