Web optimised PDF

This article is part of the Tips & Tricks series.

Installation • Usage • PDF issues • Other

Scribus' PDF is optimized for print shops and Scribus takes pains to ensure that the printed output will look identical on different printing presses. The price for this consistency is a file size several times larger than straight-forward PDF would have. This tip shows a way to reduce the file size so you can put the PDF file on the web or even distribute it by e-mail.

My wife edits and lays out a newsletter that, being mostly distributed via e-mail and the web, must be below ~ 1 MB, desirably even smaller. Since the Scribus options for minimizing the size (subsetting fonts and downsampling images) did not meet that requirement, I looked for a route to PostScript and back to PDF that would give better compression, and the following does a surprisingly good job:

1. Export as PDF (1.3 or 1.4), embedding all fonts, no font subsetting, no image subsampling → newsletter_scribus.pdf [~2.8 MB]
2. Convert to PS using `pdftops -level3' → newsletter.ps [huge]
3. Convert back to PDF using ghostscript (subsetting fonts, subsampling images) → newsletter_compact.pdf [~500 kB]. (You can use ps2pdf13 [or ps2pdf14] for this step; a more fine-grained solution is to use my script below.)
4. If you feel like it, use pdfopt to linearize the PDF, so Acroread can start showing the first pages while the rest is still being downloaded (I have never tried this feature).

Notes

• As you see, the resulting PDF is more than 5 times smaller.
• For step 2, Acroread's print-to-file PS export will not work ― you need to use Xpdf/pdftops.
• For step 3, you need a 8.x version of ghostscript; I have successfully tried AFPL 8.15 and 8.53 and GNU 8.16, while ESP 7.07 does not work.
• Marking and searching of text works fine in the new file, while there were many gaps between letters in the original (due to the way Scribus ensures precise text placement).
• I also found that the compressed PDF file looks much nicer in Acroread 7 (but not in Xpdf or Gv), but your mileage may vary:
  • True-type fonts were not antialiased in the original PDF, but were in the compressed one. However, in simple test documents, antialiasing works for both PDF files, so I don't know what the problem was in the first place.
  • Images looked too dark in the original (apparently a transparency bug in Acroread 6 and 7), but are fine in the compressed file. Again, I have some difficulties in reproducing this in a simple test document.
• There is one downside: As a result of the transformation, in Xpdf non-ascii letters (like `é' or `½') get lost. Acroread 7 or Gv have no such problem, so it might be a bug in Xpdf/Poppler. For our newsletter, the number of readers using Xpdf is close to one (myself), so this is not much of an issue.
• Also, the transformation loses meta information (creation date, creator, ...), bookmarks, PDF annotations, hyperlinks, etc. The script below fills in some of the meta information itself; I have tried to extract and restore bookmarks, but the resulting PDFs caused trouble with Acroread 7.

Additional information

• If anybody is willing to run the resulting file through a PDF checker, you can try the latest newsletter at http://www.calgarymulticulturalcentre.com/CC_January_2006.pdf

The following Perl script calls, in sequence

1. pdftk to extract some meta information
2. pdftops
3. gs
4. pdfopt

If you lack the first or last program, you'll need to comment out the corresponding lines.

You get a usage overview with

 compress-newsletter.pl -h

and before using it in production, you should fill in your details in the lines marked with `[Insert ... here]'.

The script os not very polished, but it works for me.

compress-newsletter.pl

#!/bin/sh
#  -*-Perl-*-
# ====================================================================== #
# Run the right perl version:
if [ -x /usr/local/bin/perl ]; then
  perl=/usr/local/bin/perl
elif [ -x /usr/bin/perl ]; then
  perl=/usr/bin/perl
else
  perl=`which perl| sed 's/.*aliased to *//'`
fi

exec $perl -x -S $0 "$@"     # -x: start from the following line
# ====================================================================== #
#! /Good_Path/perl -w
# line 17

# Name:   compress-newsletter
# Author: wd (Wolfgang.Dobler@ucalgary.ca)
# Date:   03-Oct-2005
# Description:
#   Use ghostscript's pdfwrite device (à la ps2pdf) to reduce the
#   Newsletter's PDF file size, and add meta information like author,
#   date, etc.
#   The preferred route is currently:
#                 [scribus>=1.2.3]
#                        |
#                    file.pdf
#                        |
#                 [pdftops>=3.00]
#                        |
#                     file.ps
#                        |
#            [pstopdf14 (gs-gnu-8.16 or higher)]
#                        |
#                        V
#                    final.pdf
# Usage:
#   compress-newletter [-i col:gray:mono] Newsletter_big.pdf
# Options:
#   -i col:gray:mono
#   --imgres=col:gray:mono   Set resolution for downsampling color,
#                            grayscale and black-and-white images
#                            (default is 144:300:300)
#   --debug                  Be verbose and keep temporary files around
use strict;
use File::Temp qw/ :mktemp /;

use Getopt::Long;
# Allow for `-Plp' as equivalent to `-P lp' etc:
Getopt::Long::config("bundling");

my (%opts);			# Options hash for GetOptions
my $doll='\$';			# Need this to trick CVS

## Process command line
GetOptions(\%opts,
	   qw( -h   --help
	       -i=s --imgres=s
	            --debug
	       -q   --quiet
               -v   --version ));

my $debug = ($opts{'debug'} ? 1 : 0 ); # undocumented debug option
if ($debug) {
    printopts(\%opts);
    print "\@ARGV = `@ARGV'\n";
}

if ($opts{'h'} || $opts{'help'})    { die usage();   }
if ($opts{'v'} || $opts{'version'}) { die version(); }

my $quiet  = ($opts{'q'} || $opts{'quiet'}  || ''           );
my $imgres = ($opts{'i'} || $opts{'imgres'} || '144:300:300');

my ($gs,      @gsargs     ) = ('gs'     );
my ($pdftops, @pdftopsargs) = ('pdftops');
my ($pdfopt,  @pdfoptargs ) = ('pdfopt' );

my $infile = shift or die usage();
(my $root=$infile) =~ s/\.(pdf|ps).*//;
(my $outfile=$infile) =~ s/(.*)(\.(pdf|ps))/${1}_new${2}/;
my $tmpfile = mktemp("${root}.tmp_XXXXXX");


## 0. Extract all sorts of information

# Extract Scribus version, creation date, bookmarks from original PDF:
print "Running pdftk ...\n";
print STDERR "pdftk $infile dump_data output\n" if ($debug);
my $meta = `pdftk $infile dump_data output -`;
my ($creator) = ( $meta =~
		  m{InfoKey: Creator\s+InfoValue:\s*(.+)$}m
		);
$creator = 'Scribus 1.2.3' unless defined($creator);
my $datestring = extract_CreationDate($meta);
my @bookmarks = extract_bookmarks($meta);

# Extract desired image resolutions
my ($colres,$grayres,$monores) = ($imgres =~ /([0-9]+):([0-9]+):([0-9]+)/);
die "Image resolution must be of form `col:gray:mono'\n"
    unless defined($monores);

## 1. Run pdftops
push @pdftopsargs, "-level3";
my $psfile = mktemp("${root}.ps_XXXXXX");
push @pdftopsargs, $infile, $psfile;
print "Running pdftops ...\n";
print STDERR "$pdftops @pdftopsargs\n" if ($debug);
system($pdftops,@pdftopsargs);

## 2. Run gs
# a) Prepare options
push @gsargs, qw{-q -dNOPAUSE -dBATCH};
push @gsargs, '-sDEVICE=pdfwrite';
push @gsargs, '-dCompatibilityLevel=1.3';
# One of /printer, /screen, /prepress, /ebook, /default; see Ps2pdf.htm:
push @gsargs, '-dPDFSETTINGS=/screen';
push @gsargs, '-dEmbedAllFonts=true';
push @gsargs, '-dSubsetFonts=true';
push @gsargs, '-dColorImageDownsampleType=/Bicubic';
push @gsargs, "-dColorImageResolution=$colres";
push @gsargs, '-dGrayImageDownsampleType=/Bicubic';
push @gsargs, "-dGrayImageResolution=$grayres";
push @gsargs, '-dMonoImageDownsampleType=/Bicubic';
push @gsargs, "-dMonoImageResolution=$monores";
push @gsargs, "-sOutputFile=$tmpfile";
push @gsargs, "-c .setpdfwrite";

# b) Write meta information to temporary file
#my $metafile = mktemp("metainfo.tmp_XXXXXX");
my $metafile = "${root}.meta";
open(META, "> $metafile");
print META <<"DEAD_PARROT";
% Document information
[%
 /CreationDate (D:$datestring)
 /ModDate (D:$datestring)
 /Creator ($creator)
 /Title ([Insert your document title here])
 /Subject ([Insert the Subject here])
 /Keywords ([Insert key words here])
 /Author ([Insert author' nsme here])
 /DOCINFO pdfmark

% Initial view on opening the document
[/View [/Fit] % Fit page in window
 /Page 1
 % /PageMode /UseOutlines % /UseNone /UserOutlines /UseThumbs /FullScreen
 /DOCVIEW pdfmark

DEAD_PARROT

## Bookmarks. [Commented out for acroread 7.0 has problems] Currently at
## the mercy of the original bookmarks (and Scribus 1.2.2 does not allow
## to edit the bookmark names) and the encoding that pdftk understands
## (most quotation marks get mapped to `?').
## Ideally, one would write out the meta information file with
## `compress-newsletter -m CC.pdf' and use it then with
## `compress-newsletter CC.pdf'.
## % Bookmarks: @bookmarks


push @gsargs, '-f', $psfile, $metafile;
print "Running gs ...\n";
print STDERR "$gs @gsargs\n" if ($debug);
system($gs,@gsargs);

## 3. Run pdfopt
print "Running pdfopt ...\n";
print STDERR "$pdfopt @pdfoptargs $tmpfile $outfile\n" if ($debug);
system($pdfopt,@pdfoptargs,$tmpfile,$outfile);

# Some diagnostics:
system('ls', '-l', $infile, $psfile, $tmpfile, $outfile);

END {
    # Clean up even in case of an error:
    unless ($debug) {
        foreach my $file ($psfile,$tmpfile) {
	    unlink $file if (defined($file) && -f $file);
        }
    }
}


# ---------------------------------------------------------------------- #
sub extract_CreationDate {

    use POSIX qw(strftime);

    my $meta = shift;

    my ($cdate) = ( $meta =~
		    m{InfoKey: CreationDate\s+InfoValue:\s*(.+)$}m
		  );
    # Time string: need to splice in "'" after hours and minutes of time zone
    # definition. To me this looks like the technical documentation was taken
    # too literally and now applications (and Acroread 7) insist on these
    # stupid markers.
    my $datestring;
    if ($cdate =~ /[0-9]{14}/) { # managed to extract CreationDate from $meta
	$datestring = "$cdate-06'00'";
    } else {		         # Creation date unknown -- use current date
	my $tz = strftime "%z", localtime();
	$tz =~ s/([0-9][0-9])([0-9][0-9])/$1'$2'/;
	$datestring = strftime "%Y%m%d%H%M%S$tz", localtime();
    }

    $datestring;
}
# ---------------------------------------------------------------------- #
sub extract_bookmarks {

    my $meta = shift;

    my @bm;

    while ($meta =~ /^BookmarkTitle:      \s* (.*) \n
                      BookmarkLevel:      \s* (.*) \n
                      BookmarkPageNumber: \s* (.*) /xmg) {
	my ($title,$level,$page) = ($1,$2,$3);
	push @bm, "[/Title ($title /Page $page /OUT pdfmark\n";
    }

}
# ---------------------------------------------------------------------- #
sub printopts {
# Print command line options
    my $optsref = shift;
    my %opts = %$optsref;
    foreach my $opt (keys(%opts)) {
	print STDERR "\$opts{$opt} = `$opts{$opt}'\n";
    }
}
# ---------------------------------------------------------------------- #
sub usage {
# Extract description and usage information from this file's header.
    my $thisfile = __FILE__;
    local $/ = '';              # Read paragraphs
    open(FILE, "<$thisfile") or die "Cannot open $thisfile\n";
    while (<FILE>) {
	# Paragraph _must_ contain `Description:' or `Usage:'
        next unless /^\s*\#\s*(Description|Usage):/m;
        # Drop `Author:', etc. (anything before `Description:' or `Usage:')
        s/.*?\n(\s*\#\s*(Description|Usage):\s*\n.*)/$1/s;
        # Don't print comment sign:
        s/^\s*# ?//mg;
        last;                        # ignore body
    }
    $_ or "<No usage information found>\n";
}
# ---------------------------------------------------------------------- #
sub version {
# Return CVS data and version info.
    my $doll='\$';		# Need this to trick CVS
    my $cmdname = (split('/', $0))[-1];
    my $rev = '$Revision: 1.8 $';
    my $date = '$Date: 2006/02/02 09:38:52 $';
    $rev =~ s/${doll}Revision:\s*(\S+).*/$1/;
    $date =~ s/${doll}Date:\s*(\S+).*/$1/;
    "$cmdname version $rev ($date)\n";
}
# ---------------------------------------------------------------------- #

# End of file compress-newsletter