Release notes

pike fish being released to water

Releasing a pike.

pikepdf releases use the semantic versioning policy.

The pikepdf API (as provided by import pikepdf) is stable and is in production use. Note that the C++ extension module pikepdf._qpdf is a private interface within pikepdf that applications should not access directly, along with any modules with a prefixed underscore.

v1.17.0

  • Fixed a failure to extract PDF images, where the image had both a palette and colorspace set to an ICC profile. The iamge is now extracted with the profile embedded. (#108)
  • Added opt-in support for memory-mapped file access, using pikepdf.open(...access_mode=pikepdf.AccessMode.mmap). Memory mapping file access performance considerably, but may make application exception handling more difficult.

v1.16.1

  • Fixed an issue with JBIG2 extraction, where the version number of the jbig2dec software may be written to standard output as a side effect. This could interfere with test cases or software that expects pikepdf to be stdout-clean.
  • Fixed an error that occurred when updating DocumentInfo to match XMP metadata, when XMP metadata had unexpected empty tags.
  • Fixed setup.py to better support Python 3.8 and 3.9.
  • Documentation updates.

v1.16.0

  • Added support for extracting JBIG2 images with the image API. JBIG2 images are converted to PIL.Image. Requires a JBIG2 decoder such as jbig2dec.
  • Python 3.5 support is deprecated and will end when Python 3.5 itself reaches end of life, in September 2020.
  • Python 3.9beta is supported and is known to work on Fedora 33.

v1.15.1

  • Fixed a regression - Pdf.save(filename) may hold file handles open after the file is fully written.
  • Documentation updates.

v1.15.0

  • Fixed an issue where Decimal objects of precision exceeding the PDF specification could be written to output files, causing some PDF viewers, notably Acrobat, to parse the file incorrectly. We now limit precision to 15 digits, which ought to be enough to prevent rounding error and parsing errors.
  • We now refuse to create pikepdf objects from float or Decimal that are NaN or ±Infinity. These concepts have no equivalent in PDF.
  • pikepdf.Array objects now implement .append() and .extend() with familiar Python list semantics, making them easier to edit.

v1.14.0

  • Allowed use of .keys(), .items() on pikepdf.Stream objects.
  • We now warn on attempts to modify pikepdf.Stream.Length, which pikepdf will manage on its own when the stream is serialized. In the future attempting to change it will become an error.
  • Clarified documentation in some areas about behavior of pikepdf.Stream.

v1.13.0

  • Added support for editing PDF Outlines (also known as bookmarks or the table of contents). Many thanks to Matthias Erll for this contribution.
  • Added support for decoding run length encoded images.
  • Object.read_bytes() and Object.get_stream_buffer() can now request decoding of uncommon PDF filters.
  • Fixed test suite warnings related to pytest and hypothesis.
  • Fixed build on Cygwin. Thanks to @jhgarrison for report and testing.

v1.12.0

  • Microsoft Visual C++ Runtime libraries are now included in the pikepdf Windows wheel, to improve ease of use on Windows.
  • Defensive code added to prevent using .emplace() on objects from a foreign PDF without first copying the object. Previously, this would raise an exception when the file was saved.

v1.11.2

  • Fix “error caused by missing str function of Array” (#100, #101).
  • Lots of delinting and minor fixes.

v1.11.1

  • We now avoid creating an empty XMP metadata entry when files are saved.
  • Updated documentation to describe how to delete the document information dictionary.

v1.11.0

  • Prevent creation of dictionaries with invalid names (not beginning with /).
  • Allow pikepdf’s build to specify a qpdf source tree, allowing one to compile pikepdf against an unreleased/modified version of qpdf.
  • Improved behavior of pages.p() and pages.remove() when invalid parameters were given.
  • Fixed compatibility with libqpdf version 10.0.1, and build official wheels against this version.
  • Fixed compatibility with pytest 5.x.
  • Fixed the documentation build.
  • Fixed an issue with running tests in a non-Unicode locale.
  • Fixed a test that randomly failed due to a “deadline error”.
  • Removed a possibly nonfree test file.

v1.10.4

  • Rebuild Python wheels with newer version of libqpdf. Fixes problems with opening certain password-protected files (#87).

v1.10.3

  • Fixed isinstance(obj, pikepdf.Operator) not working. (#86)
  • Documentation updates.

v1.10.2

  • Fixed an issue where pages added from a foreign PDF were added as references rather than copies. (#80)
  • Documentation updates.

v1.10.1

  • Fixed build reproducibility (thanks to @lamby)
  • Fixed a broken link in documentation (thanks to @maxwell-k)

v1.10.0

  • Further attempts to recover malformed XMP packets.
  • Added missing functionality to extract 1-bit palette images from PDFs.

v1.9.0

  • Improved a few cases of malformed XMP recovery.
  • Added an unparse_content_stream API to assist with converting the previously parsed content streams back to binary.

v1.8.3

  • If the XMP metadata packet is not well-formed and we are confident that it is essentially empty apart from XML fluff, we fix the problem instead of raising an exception.

v1.8.2

  • Fixed an issue where QPDF 8.4.2 would report different errors from QPDF 9.0.0, causing a test to fail. (#71)

v1.8.1

  • Fixed an issue where files opened by name may not be closed correctly. Regression from v1.8.0.
  • Fixed test for readable/seekable streams evaluated to always true.

v1.8.0

  • Added API/property to iterate all objects in a PDF: pikepdf.Pdf.objects.
  • Added pikepdf.Pdf.check(), to check for problems in the PDF and return a text description of these problems, similar to qpdf --check.
  • Improved internal method for opening files so that the code is smaller and more portable.
  • Added missing licenses to account for other binaries that may be included in Python wheels.
  • Minor internal fixes and improvements to the continuous integration scripts.

v1.7.1

  • This release was incorrectly marked as a patch-level release when it actually introduced one minor new feature. It includes the API change to support pikepdf.Pdf.objects.

v1.7.0

  • Shallow object copy with copy.copy(pikepdf.Object) is now supported. (Deep copy is not yet supported.)
  • Support for building on C++11 has been removed. A C++14 compiler is now required.
  • pikepdf now generates manylinux2010 wheels on Linux.
  • Build and deploy infrastructure migrated to Azure Pipelines.
  • All wheels are now available for Python 3.5 through 3.8.

v1.6.5

  • Fixed build settings to support Python 3.8 on macOS and Linux. Windows support for Python 3.8 is not currently tested since continuous integration providers have not updated to Python 3.8 yet.
  • pybind11 2.4.3 is now required, to support Python 3.8.

v1.6.4

  • When images were encoded with CCITTFaxDecode, type G4, with the /EncodedByteAlign set to true (not default), the image extracted by pikepdf would be a corrupted form of the original, usually appearing as a small speckling of black pixels at the top of the page. Saving an image with pikepdf was not affected; this problem only occurred when attempting to extract images. We now refuse to extract images with these parameters, as there is not sufficient documentation to determine how to extract them. This image format is relatively rare.

v1.6.3

  • Fixed compatibility with libqpdf 9.0.0.
    • A new method introduced in libqpdf 9.0.0 overloaded an older method, making a reference to this method in pikepdf ambiguous.
    • A test relied on libqpdf raising an exception when a pikepdf user called Pdf.save(..., min_version='invalid'). libqpdf no longer raises an exception in this situation, but ignores the invalid version. In the interest of supporting both versions, we defer to libqpdf. The failing test is removed, and documentation updated.
  • Several warnings, most specific to the Visual C++ compiler, were fixed.
  • The Windows CI scripts were adjusted for the change in libqpdf ABI version.
  • Wheels are now built against libqpdf 9.0.0.
  • libqpdf 8.4.2 and 9.0.0 are both supported.

v1.6.2

  • Fixed another build problem on Alpine Linux - musl-libc defines struct FILE as an incomplete type, which breaks pybind11 metaprogramming that attempts to reason about the type.
  • Documentation improved to mention FreeBSD port.

v1.6.1

  • Dropped our one usage of QPDF’s C API so that we use only C++.
  • Documentation improvements.

v1.6.0

  • Added bindings for QPDF’s page object helpers and token filters. These enable: filtering content streams, capturing pages as Form XObjects, more convenient manipulation of page boxes.
  • Fixed a logic error on attempting to save a PDF created in memory in a way that overwrites an existing file.
  • Fixed Pdf.get_warnings() failed with an exception when attempting to return a warning or exception.
  • Improved manylinux1 binary wheels to compile all dependencies from source rather than using older versions.
  • More tests and more coverage.
  • libqpdf 8.4.2 is required.

v1.5.0

  • Improved interpretation of images within PDFs that use an ICC colorspace. Where possible we embed the ICC profile when extracting the image, and profile access to the ICC profile.
  • Fixed saving PDFs with their existing encryption.
  • Fixed documentation to reflect the fact that saving a PDF without specifying encryption settings will remove encryption.
  • Added a test to prevent overwriting the input PDF since overwriting corrupts lazy loading.
  • Object.write(filters=, decode_parms=) now detects invalid parameters instead of writing invalid values to Filters and DecodeParms.
  • We can now extract some images that had stacked compression, provided it is /FlateDecode.
  • Add convenience function Object.wrap_in_array().

v1.4.0

  • Added support for saving encrypted PDFs. (Reading them has been supported for a long time.)
  • Added support for setting the PDF extension level as well as version.
  • Added support converting strings to and from PDFDocEncoding, by registering a "pdfdoc" codec.

v1.3.1

  • Updated pybind11 to v2.3.0, fixing a possible GIL deadlock when pikepdf objects were shared across threads. (#27)
  • Fixed an issue where PDFs with valid XMP metadata but missing an element that is usually present would be rejected as malformed XMP.

v1.3.0

  • Remove dependency on defusedxml.lxml, because this library is deprecated. In the absence of other options for XML hardening we have reverted to standard lxml.
  • Fixed an issue where PdfImage.extract_to() would write a file in the wrong directory.
  • Eliminated an intermediate buffer that was used when saving to an IO stream (as opposed to a filename). We would previously write the entire output to a memory buffer and then write to the output buffer; we now write directly to the stream.
  • Added Object.emplace() as a workaround for when one wants to update a page without generating a new page object so that links/table of contents entries to the original page are preserved.
  • Improved documentation. Eliminated all arg0 placeholder variable names, which appeared when the documentation generator could not read a C++ variable name.
  • Added PageList.remove(p=1), so that it is possible to remove pages using counting numbers.

v1.2.0

  • Implemented Pdf.close() and with-block context manager, to allow Pdf objects to be closed without relying on del.
  • PdfImage.extract_to() has a new keyword argument fileprefix=, which to specify a filepath where an image should be extracted with pikepdf setting the appropriate file suffix. This simplifies the API for the most common case of extracting images to files.
  • Fixed an internal test that should have suppressed the extraction of JPEGs with a nonstandard ColorTransform parameter set. Without the proper color transform applied, the extracted JPEGs will typically look very pink. Now, these images should fail to extract as was intended.
  • Fixed that Pdf.save(object_stream_mode=...) was ignored if the default fix_metadata_version=True was also set.
  • Data from one Pdf is now copied to other Pdf objects immediately, instead of creating a reference that required source PDFs to remain available. Pdf objects no longer reference each other.
  • libqpdf 8.4.0 is now required
  • Various documentation improvements

v1.1.0

  • Added workaround for macOS/clang build problem of the wrong exception type being thrown in some cases.
  • Improved translation of certain system errors to their Python equivalents.
  • Fixed issues resulting from platform differences in datetime.strftime. (#25)
  • Added Pdf.new, Pdf.add_blank_page and Pdf.make_stream convenience methods for creating new PDFs from scratch.
  • Added binding for new QPDF JSON feature: Object.to_json.
  • We now automatically update the XMP PDFVersion metadata field to be consistent with the PDF’s declared version, if the field is present.
  • Made our Python-augmented C++ classes easier for Python code inspectors to understand.
  • Eliminated use of the imghdr library.
  • Autoformatted Python code with black.
  • Fixed handling of XMP metadata that omits the standard <x:xmpmeta> wrapper.

v1.0.5

  • Fixed an issue where an invalid date in XMP metadata would cause an exception when updating DocumentInfo. For now, we warn that some DocumentInfo is not convertible. (In the future, we should also check if the XMP date is valid, because it probably is not.)
  • Rebuilt the binary wheels with libqpdf 8.3.0. libqpdf 8.2.1 is still supported.

v1.0.4

  • Updates to tests/resources (provenance of one test file, replaced another test file with a synthetic one)

v1.0.3

  • Fixed regression on negative indexing of pages.

v1.0.2

  • Fixed an issue where invalid values such as out of range years (e.g. 0) in DocumentInfo would raise exceptions when using DocumentInfo to populate XMP metadata with .load_from_docinfo.

v1.0.1

  • Fixed an exception with handling metadata that contains the invalid XML entity &#0; (an escaped NUL)

v1.0.0

  • Changed version to 1.0.

v0.10.2

Fixes

  • Fixed segfault when overwriting the pikepdf file that is currently open on Linux.
  • Fixed removal of an attribute metadata value when values were present on the same node.

v0.10.1

Fixes

  • Avoid canonical XML since it is apparently too strict for XMP.

v0.10.0

Fixes

  • Fixed several issues related to generating XMP metadata that passed veraPDF validation.
  • Fixed a random test suite failure for very large negative integers.
  • The lxml library is now required.

v0.9.2

Fixes

  • Added all of the commonly used XML namespaces to XMP metadata handling, so we are less likely to name something ‘ns1’, etc.
  • Skip a test that fails on Windows.
  • Fixed build errors in documentation.

v0.9.1

Fixes

  • Fix Object.write() accepting positional arguments it wouldn’t use
  • Fix handling of XMP data with timezones (or missing timezone information) in a few cases
  • Fix generation of XMP with invalid XML characters if the invalid characters were inside a non-scalar object

v0.9.0

Updates

  • New API to access and edit PDF metadata and make consistent edits to the new and old style of PDF metadata.
  • 32-bit binary wheels are now available for Windows
  • PDFs can now be saved in QPDF’s “qdf” mode
  • The Python package defusedxml is now required
  • The Python package python-xmp-toolkit and its dependency libexempi are suggested for testing, but not required

Fixes

  • Fixed handling of filenames that contain multibyte characters on non-UTF-8 systems

Breaking

  • The Pdf.metadata property was removed, and replaced with the new metadata API
  • Pdf.attach() has been removed, because the interface as implemented had no way to deal with existing attachments.

v0.3.7

  • Add API for inline images to unparse themselves

v0.3.6

  • Performance of reading files from memory improved to avoid unnecessary copies.
  • It is finally possible to use for key in pdfobj to iterate contents of PDF Dictionary, Stream and Array objects. Generally these objects behave more like Python containers should now.
  • Package API declared beta.

v0.3.5

Breaking

  • Pdf.save(...stream_data_mode=...) has been dropped in favor of the newer compress_streams= and stream_decode_level parameters.

Fixes

  • A use-after-free memory error that caused occasional segfaults and “QPDFFakeName” errors when opening from stream objects has been resolved.

v0.3.4

Updates

  • pybind11 vendoring has ended now that v2.2.4 has been released

v0.3.3

Breaking

  • libqpdf 8.2.1 is now required

Updates

  • Improved support for working with JPEG2000 images in PDFs
  • Added progress callback for saving files, Pdf.save(..., progress=)
  • Updated pybind11 subtree

Fixes

  • del obj.AttributeName was not implemented. The attribute interface is now consistent
  • Deleting named attributes now defers to the attribute dictionary for Stream objects, as get/set do
  • Fixed handling of JPEG2000 images where metadata must be retrieved from the file

v0.3.2

Updates

  • Added support for direct image extraction of CMYK and grayscale JPEGs, where previously only RGB (internally YUV) was supported
  • Array() now creates an empty array properly
  • The syntax Name.Foo in Dictionary(), e.g. Name.XObject in page.Resources, now works

v0.3.1

Breaking

  • pikepdf.open now validates its keyword arguments properly, potentially breaking code that passed invalid arguments
  • libqpdf 8.1.0 is now required - libqpdf 8.1.0 API is now used for creating Unicode strings
  • If a non-existent file is opened with pikepdf.open, a FileNotFoundError is raised instead of a generic error
  • We are now temporarily vendoring a copy of pybind11 since its master branch contains unreleased and important fixes for Python 3.7.

Updates

  • The syntax Name.Thing (e.g. Name.DecodeParms) is now supported as equivalent to Name('/Thing') and is the recommended way to refer names within a PDF
  • New API Pdf.remove_unneeded_resources() which removes objects from each page’s resource dictionary that are not used in the page. This can be used to create smaller files.

Fixes

  • Fixed an error parsing inline images that have masks
  • Fixed several instances of catching C++ exceptions by value instead of by reference

v0.3.0

Breaking

  • Modified Object.write method signature to require filter and decode_parms as keyword arguments
  • Implement automatic type conversion from the PDF Null type to None
  • Removed Object.unparse_resolved in favor of Object.unparse(resolved=True)
  • libqpdf 8.0.2 is now required at minimum

Updates

  • Improved IPython/Jupyter interface to directly export temporary PDFs
  • Updated to qpdf 8.1.0 in wheels
  • Added Python 3.7 support for Windows
  • Added a number of missing options from QPDF to Pdf.open and Pdf.save
  • Added ability to delete a slice of pages
  • Began using Jupyter notebooks for documentation

v0.2.2

  • Added Python 3.7 support to build and test (not yet available for Windows, due to lack of availability on Appveyor)
  • Removed setter API from PdfImage because it never worked anyway
  • Improved handling of PdfImage with trivial palettes

v0.2.1

  • Object.check_owner renamed to Object.is_owned_by
  • Object.objgen and Object.get_object_id are now public functions
  • Major internal reorganization with pikepdf.models becoming the submodule that holds support code to ease access to PDF objects as opposed to wrapping QPDF.

v0.2.0

  • Implemented automatic type conversion for int, bool and Decimal, eliminating the pikepdf.{Integer,Boolean,Real} types. Removed a lot of associated numerical code.

Everything before v0.2.0 can be considered too old to document.