[LAD] [OT] bash vs disk activity

[Forest Bond]

Hi,

On Thu, Oct 11, 2007 at 04:43:23AM -0400, Paul Davis wrote:
> On Thu, 2007-10-11 at 11:52 +0700, Patrick Shirkey wrote:
> > On Wed, 2007-10-10 at 06:37 -0400, Forest Bond wrote:
> > > Hi,
> > > 
> > > On Wed, Oct 10, 2007 at 09:51:35AM +0700, Patrick Shirkey wrote:
> > > > On Tue, 2007-10-09 at 09:16 -0400, Forest Bond wrote:
> > > > > On Tue, Oct 09, 2007 at 01:11:38PM +0200, Lars Luthman wrote:
> > > > > > Slightly-slower-hack: add the 'sync' option to whatever script or config
> > > > > > file is used to mount the device (/etc/fstab, udev rules, KDE service
> > > > > > menu etc). That way all writes will be synchronised automatically.
> > > > > 
> > > > > It sounds like this doesn't matter much for you, but keep in mind that -o
> > > > > sync doesn't cause reads to be unbuffered, so don't go trying to benchmark
> > > > > your flash devices that way!
> > > > 
> > > > Hmm, I need speed and accuracy. 
> > > 
> > > But only for writes, no?
> > 
> > For the purpose of this script I am happy with just write but I will be
> > needing read in the near future.
> > 
> > FYI, I can connect 200 usb-storage devices to a dual cpu intel with 4 GB
> > RAM and an extra pci-usb card on the same irq as the internal usb bus. 
> > 
> > Basically I am uploading content to a couple of hundred mp3/multi media
> > players at the same time and this script is used by other people with
> > less Linux knowledge than I so I'm trying to make the progress bar as
> > accurate as possible. Otherwise there is a significant lag between when
> > the progress bar says it is finished to when the transfer is actually
> > finished. Speed is also a major issue here so I cannot use the sync
> > option as that causes way too much delay compared to async.
> > 
> > Can anyone here shed a little more light on how to monitor the disk
> > activity while transferring data?
> 
> if you use your own compiled code for the copy, you can open the
> destination files with O_SYNC. from man 2 open:
> 
>       O_SYNC The file is opened for synchronous  I/O.  Any  write()s  on
>               the resulting  file  descriptor will block the calling process
>               until the data has been physically written to the underlying
>               hardware.  See RESTRICTIONS below, though.
> 
> this avoids having the entire mount be done with the sync option.

Or, use Python instead of bash...

Python gets you a full POSIX interface, and you can work a shell from within
Python (when you need it).

Alternatively, you could whip up a very small Python/C/whatever program that you
could call from your shell script.  It could even print the percentages to
stdout or stderr, making your shell script's life easier:

$ copy-sync source dest
10%
20%
30%
40%
50%
60%
70%
80%
90%
100%

-Forest
-- 
Forest Bond
http://www.alittletooquiet.net
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 189 bytes
Desc: Digital signature
URL: <http://lists.linuxaudio.org/pipermail/linux-audio-dev/attachments/20071011/a06febf7/attachment.pgp>