Compressed Files and GDAL

Most compressed GIS data can be read on fly in apps that have a backend of GDAL using a compressed virtual file system path. Below explains in a nutshell from the GDAL Virtual File handler documentation, the important parts shared below for your convenience. These virtual paths will work with Python and R, along with QGIS, ogr2ogr, gdalinfo, etc.

/vsizip/ (.zip archives)

/vsizip/ is a file handler that allows reading ZIP archives on-the-fly without decompressing them beforehand.

/vsizip/my.zip/my.tif  (relative path to the .zip)
/vsizip//home/even/my.zip/subdir/my.tif (absolute path to the .zip)
/vsizip/c:\users\even\my.zip\subdir\my.tif
/vsizip/{/vsizip/my.zip/subdir/subzip.zip}/subdir_in_subzip/my.shp (alternate syntax for a nested .zip)

Note that in R and Python for Windows, you should use forward slashes rather then backslashes unless you escape.

/vsigzip/ (gzipped file)

/vsigzip/ is a file handler that allows on-the-fly reading of GZip (.gz) files without decompressing them in advance.

/vsigzip/my.gz # (relative path to the .gz)
/vsigzip//home/even/my.gz # (absolute path to the .gz)
/vsigzip/c:\users\even\my.gz

/vsitar/ (.tar, .tgz archives)

/vsitar/ is a file handler that allows on-the-fly reading in regular uncompressed .tar or compressed .tgz or .tar.gz archives, without decompressing them in advance.

/vsitar/my.tar/my.tif # (relative path to the .tar)
/vsitar//home/even/my.tar/subdir/my.tif # (absolute path to the .tar)
/vsitar/c:\users\even\my.tar\subdir\my.tif

/vsicurl/ (http/https/ftp files: random access)

/vsicurl/ is a file system handler that allows on-the-fly random reading of files available through HTTP/FTP web protocols, without prior download of the entire file. It requires GDAL to be built against libcurl.

Recognized filenames are of the form /vsicurl/http[s]://path/to/remote/resource or /vsicurl/ftp://path/to/remote/resource, where path/to/remote/resource is the URL of a remote resource.

Chaining

It is possible to chain multiple file system handlers.

# ogrinfo a shapefile in a zip file on the internet:
ogrinfo -ro -al -so /vsizip//vsicurl/https://raw.githubusercontent.com/OSGeo/gdal/master/autotest/ogr/data/shp/poly.zip
# ogrinfo a shapefile in a zip file on an ftp:
ogrinfo -ro -al -so /vsizip//vsicurl/ftp://user:password@example.com/foldername/file.zip/example.shp

Lots more examples and details.

Leave a Reply

Your email address will not be published. Required fields are marked *