archive

4.0 Update

The Archive library was originally written when the web was the primary use of Dart. File IO was less of a concern and the design was around having everything in memory. As other uses of Dart came about, such as Flutter, a lot of File IO operations were added to the library, but not in a very clean way.

The design goal for the 4.0 revision of the library is to ensure File IO is a primary focus, while minimizing memory usage. Memory-only interfaces are still available for web platforms.

Migrating 3.x to 4.x.

Migration quick tips:

decodeBuffer has been renamed to decodeStream in the various decoder classes.
InputStream has been renamed to InputMemoryStream.
OutputStream has been renamed to OutputMemoryStream.

Overview

A Dart library to encode and decode various archive and compression formats.

The archive library currently supports the following codecs:

Zip
Tar
ZLib
GZip
BZip2
XZ

Usage

package:archive/archive.dart

Can be used for both web and native applications.

package:archive/archive_io.dart

Provides some extra utilities for 'dart:io' based applications.

Decoding a zip file in memory

import 'package:archive/archive.dart';
                import 'dart:io';
                void main() {
                  final bytes = File('test.zip').readAsBytesSync();
                  final archive = ZipDecoder().decodeBytes(bytes);
                  for (final entry in archive) {
                    if (entry.isFile) {
                      final fileBytes = file.readBytes();
                      File('out/${file.fullPathName}')
                        ..createSync(recursive: true)
                        ..writeAsBytesSync(fileBytes);
                    }
                  }
                }

Using InputFileStream and OutputFileStream to extract a zip:

import 'dart:io';
                import 'package:archive/archive.dart';
                void main() {
                  // Use an InputFileStream to access the zip file without storing it in memory.
                  // Note that using InputFileStream will result in an error from the web platform  
                  // as there is no file system there.
                  final inputStream = InputFileStream('test.zip');
                  // Decode the zip from the InputFileStream. The archive will have the contents of the
                  // zip, without having stored the data in memory. 
                  final archive = ZipDecoder().decodeStream(inputStream);
                  final symbolicLinks = []; // keep a list of the symbolic link entities, if any.
                  // For all of the entries in the archive
                  for (final file in archive) {
                    // You should create symbolic links **after** the rest of the archive has been
                    // extracted, otherwise the file being linked might not exist yet.
                    if (file.isSymbolicLink) {
                      symbolicLinks.add(file);
                      continue;
                    }
                    if (file.isFile) {
                      // Write the file content to a directory called 'out'.
                      // In practice, you should make sure file.name doesn't include '..' paths
                      // that would put it outside of the extraction directory.
                      // An OutputFileStream will write the data to disk.
                      final outputStream = OutputFileStream('out/${file.name}');
                      // The writeContent method will decompress the file content directly to disk without
                      // storing the decompressed data in memory. 
                      entity.writeContent(outputStream);
                      // Make sure to close the output stream so the File is closed.
                      outputStream.closeSync();
                    } else {
                      // If the entity is a directory, create it. Normally writing a file will create
                      // the directories necessary, but sometimes an archive will have an empty directory
                      // with no files.
                      Directory('out/${file.name}').createSync(recursive: true);
                    }
                  }
                  // Create symbolic links **after** the rest of the archive has been extracted to make sure
                  // the file being linked exists.
                  for (final entity in symbolicLinks) {
                    // Before using this in production code, you should ensure the symbolicLink path
                    // points to a file within the archive, otherwise it could be a security issue.
                    final link = Link('out/${entity.fullPathName}');
                    link.createSync(entity.symbolicLink!, recursive: true);
                  }
                }

extractFileToDisk

extractFileToDisk is a convenience function to extract the contents of an archive file directory to an output directory. The type of archive it is will be determined by the file extension.

import 'package:archive/archive_io.dart';
                // ...
                extractFileToDisk('test.zip', 'out');

extractArchiveToDisk

extractArchiveToDisk is a convenience function to write the contents of an Archive to an output directory.

import 'package:archive/archive_io.dart';
                // ...
                // Use an InputFileStream to access the zip file without storing it in memory.
                final inputStream = InputFileStream('test.zip');
                // Decode the zip from the InputFileStream. The archive will have the contents of the
                // zip, without having stored the data in memory. 
                final archive = ZipDecoder().decodeStream(inputStream);
                extractArchiveToDisk(archive, 'out');

Changelog

4.3.0

Added multithreaded decoding to XZDecoder. Passing an XZMultithreadOptions to decodeBytes or decodeStream spreads the work over isolates, one xz block per job, and reports the result through its onDone callback. Both methods behave exactly as before when it is omitted. On a 1.1 GB archive of six blocks: 16.1 s single threaded, 8.0 s on the default three workers, 5.0 s on six.
decodeStream reading from an InputFileStream now lets each worker read its own block straight from disk, so the compressed archive never passes through the calling isolate. Decoding a 1.1 GB archive to an OutputFileStream peaks at 1.8 GB, below the 3.0 GB the single threaded path uses, while being twice as fast.
Multithreaded decoding falls back to the single threaded path on the web, where there are no isolates, and the isolate machinery is tree-shaken out of web builds entirely.
Added InputFileStream.fileBuffer, fileOffset and fileLength.
Added --x86 flag support to XZDecoder
Improved verify: true speed for XZDecoder
Improved overall decode speed for XZDecoder
Decreased RAM usage for XZDecoder
Added concatenated streams support for XZDecoder
Added crc64 wasm support (verify: true)
Added uncompressedSize getter for XZDecoder, returning the original file size before compression.
Fix: pb=4 flag range error in XZDecoder
Fix: padding for _streamStart in XZDecoder
Added throwOnError to XZDecoder.decodeBytes and decodeStream. Without it a malformed or truncated archive still returns the partial output with nothing to say it is not the whole file, which was the only decode with no way at all to report a failure. In multithreaded mode the exception is delivered to XZMultithreadOptions.onError, and setting throwOnError without an onError is now refused rather than losing the failure.
Fix: a corrupt xz block lost the part of itself that had already decoded when verify: true was used with an output that cannot be read back, such as an OutputFileStream or any multithreaded decode. The output now stops in the same place whichever way the decode was asked for.
Added XZDecoder.maxPreallocateSize, capping how large an output buffer is allocated from a size the archive itself declares. Defaults to xzDefaultMaxPreallocateSize, 2 GB natively and 256 MB on the web.
Added XZMultithreadOptions.fileReadBufferSize.
Fix: an xz --check=none archive with corrupt blocks passed verify: true.
Fix: the xz block header reader trusted its own fields, accepting an unterminated multibyte integer, a filter properties length past the end of the header, empty delta or LZMA2 properties, and a bad stream header or footer CRC.
Fix: the zip End of Central Directory record was missed when its signature straddled a chunk of the backwards search, or sat within the last 21 bytes of the file, so a valid archive could decode as empty.
TarDecoder with verify: true now checks each header's own checksum and throws ArchiveException on a mismatch.
Fix: GNU base-256 numeric tar header fields were read as octal, so sizes of 8 GB or more and large uid, gid and mtime values decoded as garbage.
Fix: tar PAX extended headers are walked by each record's declared length instead of split on newlines, so a record holding binary no longer corrupts the rest of the header.
Fix: a PAX size record was applied to the metadata headers following it rather than to the entry it describes.
Fix: a GNU ././@LongLink header of type K set the next entry's name instead of its symlink target.
Fix: storeData: false skipped the tar long name and PAX headers along with the file data, losing the names they carry.
Fix: a negative size in a tar header is refused instead of read as an entry.
Fix: with verify: true the end of a tar is a full 512 byte zero block, which two zero bytes could not be told apart from a damaged header. A tail shorter than a header block now ends the archive instead of decoding as junk.
Fix: tar base-256 fields wider than TarFile.maxNumericField are refused on read and write instead of overflowing silently.
Fix: decoded regular tar files were given an empty link name and re-encoded as symlinks.
Fix: long tar names and symlink targets were written without the L and K type flags, so other tar implementations truncated them to 100 bytes, and long symlink targets were not written at all.
Fix: the GNU long name record was sized in characters rather than encoded bytes and ignored filenameEncoding.
Fix: encoding a tar entry consumed the stream it was given, so a second encode of the same archive wrote empty entries.
Fix: tar values too wide for the octal header field are written in GNU base-256 form instead of producing a corrupt header.
TarEncoder copies a stored stream through in chunks rather than reading the whole entry into memory.
TarFile.content is now settable and reads back what was set.
Fix: GZipDecoder.decodeStream returned true for a truncated or corrupt archive, silently losing files.
Fix: the web gzip decoder never checked the member CRC32 despite verify: true.
Fix: the web gzip header reader trusted its own length fields, reading past the end on a damaged extra field, an unterminated name or comment, or a header under ten bytes.
Fix: a gzip input under 20 bytes had header bytes read as a trailer.
The native gzip decoder now reads in 8 KB chunks instead of 1 KB.
Fix: an incomplete Huffman code table entry consumed no bits and looped forever, and a code length repeat past the end of the table indexed out of range.
Fix: an Inflate back-reference distance reaching past everything written indexed behind the output buffer.
Fix: the web zlib decoder read its two byte header without checking two bytes were there.
Fix: a short read near the end of a file overwrote the buffer's size with the bytes read, so every later read missed the cache and went back to disk.
Fix: multi-byte FileBuffer reads refilled the cache one byte early, costing a re-read on every buffer-aligned access.
Fix: InputFileStream.subset did not clamp its length to what remains of the source, so peeking past the end returned stale bytes.
Deprecated the fileSize parameter of the FileBuffer read methods, which is ignored.

4.2.0

Optimize performance and issues with large files with XZDecoder.

4.1.0

Fix: RangeError when parsing ZIP extra fields with trailing bytes.
Optimize LZ77 decode
Fix: issue with file_buffer preventing last value
Fix: invalid path separator in zip content compression

4.0.9

Fix extractFileToDisk file extension handling, where foo.bar.zip would fail.

4.0.8

Remove dependency to crypto package
Removed Adler32 and Crc32 classes
Fix: extractArchiveToDisk extensions should be case insensitive.
Fix: extractFileToDisk for .tar.gz files
Add ZipFileEncoder.addDirectorySync
Always use posix separators for ZipFileEncoder

4.0.7

Change posix dependency to 6.0.2.

4.0.6

Fix zip decoding when the last file of the archive is also a zip.
Add lastModifiedDateTime to ArchiveFile
Fix Archive files and fileMap getting out of sync after calling removeFile

4.0.5

Improve performance of OutputFileStream.
Add ArchiveFile.noCompress, which had been removed from the 3.x to 4.x update.
GZipDecoder should fall back to ZLib if there is no GZip header.

4.0.4

Fix level argument for ZipEncoder.add method.
Fix ArchiveFile.compression to work for controlling compression method used encoding zips.
Add ArchiveFile.compression CompressionType.bzip2 compression mode support.
Add ArchiveFile.compressionLevel to work for controlling compression level used for encoding zips.

4.0.3

Fix potential infinite loop when parsing zip headers.
Update conditional imports to be compatible with WASM.
Add addFileSync to ZipFileEncoder for synchronous call to addFile.

4.0.2

Reduce SDK min version to 3.0.
Fix import error with js_interop.

4.0.1

Fix error with GZip encoder for web builds.

4.0.0

Major cleanup of the code, includes potential breaking changes.
- decodeBuffer has been renamed to decodeStream in the various decoder classes.
- InputStream has been renamed to InputMemoryStream.
- OutputStream has been renamed to OutputMemoryStream.

3.6.1

Fix ArchiveFile.rawContent returning null after decoding a zip.

3.6.0

Fix zip encoding when a file was previously decoded.
Fix decoding zips with password when using InputFileStream.
ZipEncoder.encode autoClose now defaults to false.

3.5.1

Re-add zipPath to ZipFileEncoder.

3.5.0

Remove dependency to pointycastle package
Use utf8 encoding for string data
Fixes for encrypted zip encoding
Async and sync versions of extractArchiveToDisk

3.4.10

Fix ZipCrypto decryption

3.4.9

Revert breaking change for extractArchiveToDisk becoming async; add extractArchiveToDiskAsync for the async version.

3.4.8

Improve zip decompression performance with dart:io by using native ZLib decompression when possible.

3.4.7

Improve performance by not using List.setRange for copying bytes, which turns out to be very slow.

3.4.6

Fix for Zip64 file size causing memory errors.

3.4.5

Rewrote InputFileStream to reduce overall memory by using a shared file cache.
Added DateTime lastModDateTime getter to ArchiveFile.
Add support for zip encryption.

3.4.4

Fix for new default buffer size for InputFileStream consuming too much memory for large archives.

3.4.3

Fix bug in InputFileStream that caused it to only have an 8-byte buffer, making file streaming slow.
Increase the default buffer size for file I/O streams to 1MB.
Update pubspec dependency versions.

3.4.2

Add bzip2 decompression for zip files.

3.4.1

Fix for decoding zip64 zip files that have multiple extra fields.

3.4.0

Add Zip64 support to ZipEncoder to allow it to create zip files > 4GB.

3.3.9

Fix for extractFileToDisk causing corrupt files by closing a file stream before it finished writing.

3.3.8

Fix for zip security issue with symlinks, https://github.com/brendan-duncan/archive/issues/265. https://osv.dev/vulnerability/GHSA-9v85-q87q-g4vg.
Fix for zip security issue with file paths, https://github.com/brendan-duncan/archive/issues/266. https://osv.dev/vulnerability/GHSA-r285-q736-9v95.
Add progress callback for decoding zip files.
Don't allow tar files to include absolute paths.
Fix error decoding AES-192.

3.3.7

Add Zip AES-256 decryption
Fix symlink encoding for tar files

3.3.6

Fix errors decoding XZ files.

3.3.5

Fix file content when decoding zips

3.3.4

Fix analysis errors.

3.3.3

Support symlinks in ZIP archives
Fix ZIP decryption for ZipCrypto format

3.3.2

Fix for UTF-8 file name caused problem on Windows.

3.3.1

Fix for Inflate crashing on some compressed files.

3.3.0

IO encoders (ZipFileEncoder, TarFileEncoder), will now include directories and empty directories.
Fix for ZipEncoder file lastModTime.
Fix for ArchiveFile.string.
Add PAX format to tar decoder.
Make more file operations async.

3.2.2

Re-add List content data for ArchiveFile.
Add String and TypedData (Int32List, Float32List, etc) content data for ArchiveFile.

3.2.1

Added buffer to OutputFileStream to improve performance by reducing the number of file writes.

3.2.0

For non-web applications, use native 'inflate' decompression when decompressing zip files.
Add asyncWrite option to extractArchiveToDisk and extractFileToDisk, moving file write operations to be async.
ArchiveFile.writeContent will release its memory after the data has been written, reducing overall memory usage.
Add clear method to ArchiveFile, clearing any decompressed data memory it's storing.

3.1.11

Fix indexing bug in Archive.addFile.

3.1.10

Fix performance regression with Archive.

3.1.9

Fix FileInputStream to work with ZipDecoder.

3.1.8

Catch invalid UTF8 string decoding.

3.1.7

Fix for UTF8 filenames

3.1.6

Fix problem with non-terminating long filenames.
File modification dates were incorrectly stored in milliseconds instead of seconds.

3.1.5

Disable XZ format CRC64 for html builds to fix errors.

3.1.4

Changed LICENSE to MIT.

3.1.3

Cleaned up LICENSE, moving other licenses to LICENSE-other.md.

3.1.2

Added the ability to override the timestamp encoded in a Zip file.

3.1.1

Fix zip encoder so that zip files created on Windows will open correctly on Linux.

3.1.0-dev

Added const constructors to ZLibDecoder, ZLibEncoder, and ZLibDecoderBase.

3.0.0

Stable release supporting null safety.

3.0.0-nullsafety.0

Migrate to null safety.

2.0.13

Switch to dart strong mode; refactor code to resolve all dartanalyzer warnings.

2.0.12

Fix dartanalyzer warnings

2.0.11

Set the default permission for ArchiveFile to something more reasonable (0644 -rw-r--r--)

2.0.10

Fix for decoding empty zip files.

2.0.9

Add isSymbolicLink and nameOfLinkedFile to ArchiveFile.
Fix for encoding empty files.

2.0.8

Fix zip isFile

2.0.7

Fix zip file attributes.

2.0.6

Support GNU tar long file names
Maintain unix file permissions in zip archives.

2.0.5

Use dart:io ZLibCodec when run from dart:io.

2.0.4

Fix InputStream when a Uint8ListView is used as input data.

2.0.3

Use Utf8 for reading strings in archive archive files, for filenames and comments.

2.0.2

Fixes for ZipFileEncoder.

2.0.1

Remove the use of part and part of in the main library.
Added ZipFileEncoder to encode files and directories using dart:io.
Added createArchiveFromDirectory function to create an Archive object from a dart:io Directory.

2.0.0

Moved version up for Dart 2 support.
Fixed an issue with file compression flags when decoding zip archives.
Fixed an issue with bzip2 decoding in production code.

1.0.33

Support the latest version of package:args.

1.0.30

Add archive_io sub-package for supporting file streaming rather than storing everything in memory. This is a work-in-progress and under development.

1.0.29

Fix issue with POSIX tar files.
Upgrade dependency on archive to >=1.0.0 <2.0.0

1.0.20

Improve performance decompressing large files in zip archives.

1.0.19

Disable CRC verification by default when decoding archives.

1.0.18

Add support for encoding uncompressed files in zip archives.

1.0.17

Fix a bug in InputStream.

1.0.16

Add stream support to Inflate decompression.

1.0.15

Improved performance when writing large blocks.

1.0.14

Misc updates and fixes.

1.0.13

Added BZip2 encoder.
BREAKING CHANGE: File was renamed to ArchiveFile, to avoid conflicts with dart:io.

1.0.12

Added BZip2 decoder.

1.0.11

Changed InputStream to work with typed_data instead of List<int>, should reduce memory and increase performance.

1.0.10

Renamed InputBuffer and OutputBuffer to InputStream and OutputStream, respectively.
Added readBits method to InputStream.

archive

Readme

archive

4.0 Update

Migrating 3.x to 4.x.

Migration quick tips:

Overview

Usage

Decoding a zip file in memory

Using InputFileStream and OutputFileStream to extract a zip:

extractFileToDisk

extractArchiveToDisk

Changelog

4.3.0

4.2.0

4.1.0

4.0.9

4.0.8

4.0.7

4.0.6

4.0.5

4.0.4

4.0.3

4.0.2

4.0.1

4.0.0

3.6.1

3.6.0

3.5.1

3.5.0

3.4.10

3.4.9

3.4.8

3.4.7

3.4.6

3.4.5

3.4.4

3.4.3

3.4.2

3.4.1

3.4.0

3.3.9

3.3.8

3.3.7

3.3.6

3.3.5

3.3.4

3.3.3

3.3.2

3.3.1

3.3.0

3.2.2

3.2.1

3.2.0

3.1.11

3.1.10

3.1.9

3.1.8

3.1.7

3.1.6

3.1.5

3.1.4

3.1.3

3.1.2

3.1.1

3.1.0-dev

3.0.0

3.0.0-nullsafety.0

2.0.13

2.0.12

2.0.11

2.0.10

2.0.9

2.0.8

2.0.7

2.0.6

2.0.5

2.0.4

2.0.3

2.0.2