2020.04.19 17:01 "[Tiff] tiffsplit from FIFO", by Levi Wilson
Curious if anyone has any experience processing larger documents with named pipes. The context that I'm working in is that I'd like to process a TIFF file from S3, but start processing the file as it is downloading. This could be useful in a lambda execution environment where space is limited.
I was thinking that I could do the following:
mkfifo fifo.tif
tiffsplit fifo.tif &
aws s3 cp s3://my-bucket/some/path/to/my_image.tif fifo.tif
When I do this, I start up tiffsplit and it seems to be waiting, but when I fire off the aws s3 cp command I get this:
$ tiffsplit fifo.tif
TIFFFetchDirectory: fifo.tif: Seek error accessing TIFF directory.
TIFFReadDirectory: Failed to read directory at offset 15278.
And on the writing side I get this:
$ aws s3 cp s3://my-bucket/some/path/to/my_image.tif fifo.tif
download failed: s3://traverse-levi/multi_image_content_items/016/aa2/08-/016aa208-003a-41af-968b-2caeb6c718e0_files to ./fifo.tif [Errno 32] Broken pipe
I'm probably doing something wrong with this, but curious if anyone else has attempted something like this.