2020.04.19 17:42 "Re: [Tiff] tiffsplit from FIFO", by Even Rouault

Levi,

Personnally, I would do that using GDAL (which I happen to be one of the developers), a free & open-source library and set of tools, that has network-based "virtual" file systems, and in particularly to handle S3, and is TIFF capable (using libtiff)

See in particular /vsis3/ at:

https://gdal.org/user/virtual_file_systems.html#vsis3-aws-s3-files-random-reading

Of use for you would be the gdal_translate utility, with its -sds mode:

https://gdal.org/programs/gdal_translate.html#cmdoption-gdal-translate-sds

For your use case, you'd do:

gdal_translate -sds /vsis3/my-bucket/some/path/to/my_image.tif split.tif --config CPL_VSIL_CURL_ALLOWED_EXTENSIONS ".tif"

and it will generate split_1.tif, split_2.tif, etc... without locally downloading the file.

You can also extract, let's say the second page, with something like:

gdal_translate GTIFF_DIR:2:/vsis3/my-bucket/some/path/to/my_image.tif split_2.tif --config CPL_VSIL_CURL_ALLOWED_EXTENSIONS ".tif"

(--config CPL_VSIL_CURL_ALLOWED_EXTENSIONS ".tif" is just to enhance performance for non-geospatial applications, to avoid any attempt to access "side-car" files)

/vsis3/ is indeed used a lot in lambda executation environments.

Community channels for GDAL at: https://gdal.org/community.html

Another possibility on Unix operating systems is to use

https://github.com/s3fs-fuse/s3fs-fuse

You could likely use tiffsplit directly with it.

Even

Spatialys - Geospatial professional services
http://www.spatialys.com