[Cialug] Efficiently removing the beginning of a file

[Chris Freeman]

I like 'tail' where 'dd' doesn't work properly. It might not be as fast as
an optimal 'dd', but it can't be worse than your experience. I don't have a
good place to test right now. Sorry.

Chris

On 5/21/07, Daniel A. Ramaley <daniel.ramaley at drake.edu> wrote:
>
> I have a ~70 MB file. The first 3635 bytes need to be removed. What is
> the most efficient way to do that? I did this, knowing it would work
> but would be slow:
>     $ dd if=inputfile of=outputfile ibs=1 obs=1M skip=3635
> It did indeed work. But it took 274 seconds (and pegged the CPU the
> entire time), whereas simply copying the file with cp only takes 2
> seconds. Since what i want to do is not *that* different an action from
> just copying the file (at least in terms of the minimum disk operations
> that would be required), it seems to me that there should be a way to
> do it that only takes ~2 seconds. What are some other command line ways
> to do this that would be more efficient?
>
> Actually, before hitting send i tried another test, just flipping the
> "ibs" and "skip" values:
>     $ dd if=inputfile of=outputfile2 ibs=3635 obs=1M skip=1
> That only took 2.5 seconds, which is much closer to the theoretical 2
> second time that should be possible. But i guess what i'm curious about
> is the general problem; if there is a large file and you need to remove
> some small number of bytes from the beginning of it, how is that best
> accomplished? If i had needed to remove only 1 byte for example, i
> would have had to have used "ibs=1 skip=1" which would have taken
> around 274 seconds again.
>
> ------------------------------------------------------------------------
> Dan Ramaley                            Dial Center 118, Drake University
> Network Programmer/Analyst             2407 Carpenter Ave
> +1 515 271-4540                        Des Moines IA 50311 USA
> _______________________________________________
> Cialug mailing list
> Cialug at cialug.org
> http://cialug.org/mailman/listinfo/cialug
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://cialug.org/pipermail/cialug/attachments/20070521/c20ce83c/attachment.htm