PDQ (Pretty Damn Quick) Software Distribution

PDQ (Pretty Damn Quick) Software Distribution

This page was last edited on Jun 28, 2016

Current Distribution

Purpose:       What is PDQ?
Version:       6.2.0
Build:       082015
Languages:       C, Perl, Python, R, (Legacy: Java, and PHP)
Platforms:       Linux, OS X, and Windows (Legacy: Solaris)
Maintainers:       Neil Gunther, Paul Puglia
Contributors:       Denny Chen, Phil Feller, Neil Gunther, Peter Harding, Paul Puglia, Sam Zallocco


1  Version 7.0.0 (2016)
2  Version 6.2.0 (2015)
    2.1  Download by Language
    2.2  Report Banner
    2.3  Circuit Existence
    2.4  PPA Book Models
    2.5  Local Variables
    2.6  Reminders
3  Version 6.1.1 (2013)
    3.1  Building PDQ - A New Approach
        3.1.1  Perl5 PDQ
        3.1.2  Python PDQ
        3.1.3  R PDQ
    3.2  Changes to PDQ Report Layout
        3.2.1  The Header
        3.2.2  Workload Parameters
        3.2.3  Resource Parameters
4  Version 6.0.1 (2012)
        4.0.4  Changes to PDQ C-library for PDQ-R
        4.0.5  Special note on PDQ-R with Windows
        4.0.6  Beta Testing
5  Version 5.0.4 (2011)
6  Version 5.0.3 (2009)
7  Version 5.0.2 (2009)
8  Version 5.0.1 (2009)
9  Version 4.2.1 (2007)
10  Version 4.0.1 (2007)
11  C Language PDQ (1998, 2000)
12  Online User Guide
13  Get Notified

1  Version 7.0.0 (2016)

Major release is planned. Check the Wiki on GitHub for development progress.

2  Version 6.2.0 (2015)

Minor release 6.2.0, build 082015.

2.1  Download by Language

You can download just those files that are necessary to build PDQ in your preferred language. Clicking any of the following links will automatically start a download of the tarball file from SourceForge.
  1. R distribution
  2. Python distribution
  3. Perl5 distribution
Otherwise, the complete source code distribution can be downloaded and installed in the usual way.
This separated packaging also facilitates the installation of PDQ on the most common operating systems
  1. Linux
  2. OS X
  3. Windows
For more detailed information about installing Perl PDQ, Python PDQ or R PDQ on Windows (where ancillary language support is needed), please see the relevant release notes for PDQ 6.1.1 (Section 3) and PDQ 6.0.1 (Section 4).
Mac OS X also needs the clang compiler from XCode tools.

2.2  Report Banner

The 6.2.0 Report banner that displays the name of your model, the date and time your PDQ model was solved, and the version of PDQ used, has been tweaked slightly to look like this:
                        PRETTY DAMN QUICK REPORT         
               ***  on   Thu Aug 20 15:33:21 2015     ***
               ***  for  Get Nopt Test                ***
               ***  PDQ  Version 6.2.0 Build 082015   ***

2.3  Circuit Existence

PDQ 6.2.0 now tests for the existence of queueing nodes, workload streams, and service demands, instantiated respectively by CreateNode(), CreateOpen() or CreateClosed(), and SetDemand() as part of defining a PDQ circuit. If any of these necessary components is missing, a warning message appears. In the next major release, execution will terminate.

2.4  PPA Book Models

The C language PDQ models in ../examples/ppa_1998, pertaining to The Practical Performance Analyst (PPA) book, were not building due to lack of source and object dependencies in the Make file. This has been fixed in PDQ 6.2.0.

2.5  Local Variables

Initialized the local variables in PDQ 6.2.0 utility functions to suppress ANSI compiler warnings.

2.6  Reminders

Don't forget
  1. the online PDQ manual. See Section 12 below.
  2. how to get notified about PDQ releases. See Section 13 below.
  3. instruction on how to use PDQ.

3  Version 6.1.1 (2013)

Minor release 6.1.1, build 011013. This minor release (coming so soon after 6.0.1) was motivated by our desire to make more of the PDQ language extensions available on Windows; not just PDQ-R. This is particularly true for Perl, since that is the language used for all the PDQ models in the Perl::PDQ book.
We realized shortly after the release of 6.0.1 that the compilation procedure we developed for R could also be used as a template for re-packaging the other language extensions. In other words: packaging the code as a source tarball, building it with the language's native compiling facilities, and making each of them available to run under Windows. The specific details for building the full 6.1.1 distribution, along with each of the re-packaged extensions, are described below, beginning in Section 3.1.
The other minor change pertains to a reformatting of some sections of the output generated by the PDQ Report() function. See Section 3.2.
To acquire the full PDQ 6.1.1 distribution, download the zipped tarball from Sourceforge. The tarball now shows major, minor and patch release numbers. On *nix systems, make sure you have root privileges to avoid permissions problems.
Very important note. Read and heed:
If you only want to build PDQ specifically for one of the particular languages: Python or R or Perl—especially in a Microsoft Windows environment—goto Section 3.1.
Do not cherry-pick your favorite language by just running make from within that sub-directory of the full distribution without running the top-level make. Otherwise, PDQ will not get compiled.
Whether you are a new or returning PDQ user, please join the Guerrilla google group and follow The Pith of Performance blog to keep abreast of all the latest developments.

3.1  Building PDQ - A New Approach

In this release, two major changes have been made to the build process for the full distribution.
As in previous versions of PDQ, the build process for the full distribution is started by running make as root from the top-level directory. See the README file in the untar-ed download for full details.
Each of the three language extension packages shipped with PDQ (Perl5, Python, and R) have been reconfigured so they build using the "offical" compile method for their language (e.g. ExtUtils::MakeMakers for Perl.) This continues (and finishes) the process we started in 6.0.1 where we provided the R language extension as its own separate source package. By doing this, we've removed the need to build the full distribution in order to compile the extensions. All three extensions will now be available as separate, native source packages that can be built outside of the full distribution. And, like R, we'll be making each of them available in the PDQ download area on Sourceforge.
The following sections gives specifics on building and installing each of the three language extensions using their respective source packages.

3.1.1  Perl5 PDQ

The PDQ Perl5 Extension has now been re-packaged as Perl source module that can be compiled using the ExtUtils::MakeMaker extension. While this greatly reduces the complexity of installing the extension on all platforms, on Windows, using the most current versions of Strawberry and ActiveState Perl, it provides a relatively staightforward means of making the extension available on that platform, without the need to also install a separate Unix/Linux-like environment such as Cygwin or MinGW.
The next two subsections provide details on how to install the Perl Extension for both Unix-like and Windows operating systems.
Unix, Linux, MacOS X:
  1. Download and unpack the tarball pdq-6.1.1.tar.gz using this PERL source link.
  2. Change directory into the pdq-6.1.1
  3. Execute the following commands:
    1. perl Makefile.PL
    2. make
    3. make test—a PDQ report will be displayed if the execution of test.pl is successful
    4. make install
The Perl PDQ module has been successfully built and tested using both Strawberry Perl and ActiveState Perl. Strawberry Perl ships with all the necessary compilers need for installing, and, if this is the version of Perl you're running, you can go immediately to step 1 below.
If you're running ActiveState Perl, you need to install two addition ActiveState PPMs before the build process can proceed. You can do this by using the following two commands: With both of these PPMs installed, you can now go to step 1 below.
Assuming all the prerequisites have already been met, you can build the Perl extension as follows:
  1. Download and unpack the tarball pdq-6.1.1.tar.gz using this PERL source link.
  2. Open a CMD window as Administrator
  3. Change directory into the pdq-6.1.1
  4. Execute the following commands:
    1. Perl Makefile.PL
    2. dmake (Notice it's dmake, NOT make)
    3. dmake test—a PDQ report will be displayed if the execution of test.pl is successful
    4. dmake install

3.1.2  Python PDQ

Like the Perl extension, the Python extension has been reconfigured to build separately using the Python's Distutils. This should make the install process significantly easier for Linux,Unix and/or MacOS X users who are only interested in using this extension and have no interest in other parts of PDQ. At this point, there is no offical support for Windows. Technically, the extensions should build, but unlike Perl and R that ship with their on compiler environments for Windows, building Python on Windows requires using Visual C/C++ tools and no testing with this compiler has been done yet. We'd be interested in hearing about the experiences of users who try compiling on Windows.
The default procedure for building the Python extension is:
  1. Download and unpack the tarball pdq-6.1.1.tar.gz using this PYTHON source link.
  2. Change directory to pdq-6.1.1
  3. The command: python setup.py install will build and install the package on most platforms, if it is run with root priviledges. On MacOS X, this can also be accomplished by prepending the sudo command as:
    sudo python setup.py install
    If you don't have root access on the install machine, a "local" compile can be included in your PYTHONPATH with the command:
    python setup.py install -user
  4. You can test whether or not the install is working correctly by running the test.py script located in the pdq-6.1.1 directory.

3.1.3  R PDQ

As of PDQ release 6.0.1, the PDQ-R extensions has been supplied as a separate source tar. It has been sucessfully installed on Linux, MacOS X and Windows. Building PDQ-R on Microsoft Windows is fairly simple process, even for those R Windows users who don't regularly build packages from sources. The steps are as follows:
  1. Download and install the R tools for Windows package from CRAN
  2. Download the PDQ-R source tarball pdq_6.1-1.tar.gz from the Sourceforge/Files page into a local directory e.g. C:\Users\Example\R-SRC
    Note the underscore in the tarball file name, which should not to be confused with the full PDQ distribution tarball.
  3. Start your version of R
  4. Use the command install.packages from the R console to install the package. The command will be something like this:
  5. For the source directory shown in the earlier example, the complete command would be:

3.2  Changes to PDQ Report Layout

Changes have been made to the following sections in the output generated by the PDQ Report() function:
  1. Header section
  2. Workload Parameters section
  3. Resource Parameters section

3.2.1  The Header

The 6.1.1 Report header has been slimmed down to look like this:
                        PRETTY DAMN QUICK REPORT         
               ***    of: Mon Feb 11 11:54:52 2013    ***
               ***   for: M/M/1 queue                 ***
               ***   Ver: PDQ Analyzer 6.1.1 011013   ***

The surrounding borders have been simplified so as to consume less vertical space. The header width has been increased to accommodate the complete 3-level version numbering. Previously, only 2 levels were shown. The section title is now displayed in all caps to be consistent with other sections.

3.2.2  Workload Parameters

Single nodes, invoked with the CreateNode() function, are identified by the FCFS (first come first served) scheduling class while multiserver queues, invoked with the CreateMultiNode() function, are identified by the MSQ (multi server queue) scheduling class.
Node Sched Resource   Workload   Class     Demand
---- ----- --------   --------   -----     ------
  1  FCFS  Select     Calls      Open      0.5000
  3  MSQ   Claims     Calls      Open      3.3420
  7  MSQ   Policy     Calls      Open      9.2228

The Node column shows the number of servers available to each resource. That parameter now also reappears in the Resource section for easier cross-referencing.

3.2.3  Resource Parameters

Two new rows, identified by the following resource metrics:
  1. Capacity metric
  2. In service metric
have been added to this section. The new format looks like this:
Metric          Resource     Work              Value   Unit
------          --------     ----              -----   ----
Capacity        Policy       Calls                 7   Servers
Throughput      Policy       Calls            0.5833   Calls/Mins
In service      Policy       Calls            5.3800   Calls
Utilization     Policy       Calls           76.8567   Percent
Queue length    Policy       Calls            6.7826   Calls
Waiting line    Policy       Calls            1.4027   Calls
Waiting time    Policy       Calls            2.4046   Mins
Residence time  Policy       Calls           11.6274   Mins

The Capacity metric (1st row) indicates the number of servers available at that resource and is a repeat of the Node value in the Workload Parameters section. This can be used to check the intent of your PDQ model.
The In service metric (3rd row) refers to the average number of requests being served. It also indicates the total utilization of multi-server capacity.
In the above example, an average of 5.38 Policy servers out of a possible 7 servers are being utilized by the Calls workload. For a single server (i.e., a PDQ node with a Capacity Value of 1), the numerical value of the In service metric should be identical to the numerical value of the Utilization metric (4th row), except that the latter is expressed as a percentage (as it was in previous releases). Once again, this can be used as a cross check.

4  Version 6.0.1 (2012)

Major release 6.0.1, build 121512. If you are a new (or returning) user of PDQ, please join the Guerrilla google group and follow The Pith of Performance blog to keep abreast of all the latest developments.
Download the zipped tarball from Sourceforge. The tarball only shows major and minor 6.0 release numbers.
In a *nix shell, be sure to issue a sudo make command to avoid permissions problems.
The main purpose of this release is improved compatibility and stability between PDQ and the R statistical environment. Many of the PDQ models, previously found in the ../examples/ directory, can now also be accessed via the demo() command in the R-console.
After installing PDQ, issue the following commands in the R-console:
> library(pdq)
> demo(package="pdq")

and a list of available PDQ-R models will be shown. These PDQ-R scripts can be executed and studied entirely from within the R environment in the usual way. Use the help() function for an online introduction:
> help(pdq)

To get a listing of PDQ function calls, go to the bottom of the Help page in R and click on the Index link. Too see the R source code for the PDQ-R models, click on the Code demos link at the top of the PDQ listings page.
Testing was carried out using R version 2.15.2 (2012-10-26). Operationally, PDQ (of any flavor) should appear cosmetically the same as the release 5 version; no additional programming required. Further background information about this release can be found in the blog posts: PDQ 6.0 is On Its Way and PDQ 6.0 from a Developer Standpoint.

4.0.4  Changes to PDQ C-library for PDQ-R

The following changes to the PDQ C-library were made to better support PDQ-R: · Replaced printf, fprintf functions with macro PRINTF to support toggling between c-library functions and R API Rprintf() functions. This routes output to the R console instead of directly to stdout. The chief benefit of this that the R sink() function can now redirect the output of the PDQ-R Report() function.
· Added code as an #ifdef which toggles between standard c-library calloc() and free() and R API Calloc() and Free() functions. R API functions are used when compiling PDQ source files inside an R package. This allows PDQ to use R memory management APIs when running inside R.
· Added code as an #ifdef that toggles between standard c-library exit() and R API error() functions. This fixes the issue of PDQ-R causing the R console to end abruptly when an error condition is hit. Error messages are also redirected to the R console.
· Twenty six (26) demo scripts have been added to the PDQ-R package. A full listing of scripts can be seen by running demo(pdq) after the R package has been loaded into your environment using the command, library(pdq).
· With this release, PDQ-R is also being provided as an R source tar.gz. This tar ball is exactly the same code that the build process of the distribution produces. This should provide PDQ-R users who want to run package on Microsoft Windows a means of building and installing the package without the need to build the entire PDQ distribution.

4.0.5  Special note on PDQ-R with Windows

Building PDQ-R on Microsoft Windows is fairly simple process, even for those R Windows users who don't regularly build packages from sources. The steps to do are as follows:
· Download and install the R tools for Windows package from CRAN
· Download the PDQ-R source tar ball from the Sourceforge/Files page into a local directory (e.g. C:\Users\Example\R-SRC)
· Start your version of R
· Use the command install.packages from the R console to install the package. The command will be something like this:
· For the source directory shown in the earlier example, the complete command would be:

4.0.6  Beta Testing

Thanks to Denny Chen and Ed Borasky for helping to test the beta release.

5  Version 5.0.4 (2011)

Release 5.0.4 Build 030211.
  1. Fixed a bug that was identified on the GCaP forum. For models with multiple streams (workloads) on multiple nodes (queues), it was possible for the arrival rate of stream-B to be reported as exceeding the saturation throughtput of say, node-X, but using the (inverse) service demand of stream-A on node-X in the calculated comparison. In other words, the index cross-referencing was wrong. This problem only showed up at or above saturation thresholds, which is why it managed to escape previous detection in the QA test suite.
  2. Improved error messaging to show explicit stream and node names when saturation threshold is exceeded. For example, in the above case:
    ERROR in procedure 'canonical()': Arrival rate 34.560 for stream 'workB' exceeds saturation thruput 34.483 of node 'CPU' with demand 0.029
  3. Updated PDQ online manual synopsis for SetTUnit and SetWUnit. Must call CreateOpen or CreateClosed before calling SetWUnit or SetTUnit.
  4. Don't forget, you need to perform a sudo make in the top-level /pdq directory to ensure write permission for the various support directories.

6  Version 5.0.3 (2009)

Release 5.0.3 Build 071209.
  1. This release ensures that all PDQ functions are available in each programming-language environment and fixes a bug in how mixed workloads are calculated, as well as a bug with displayed device utilizations in PDQ Report().
  2. PDQ can be built and installed by issuing a sudo make command in the top-level /pdq directory. The sudo is important to ensure you the right permissions to write into the various support directories. This is especially important for the R build to work.
  3. An as yet unresolved problem with PDQ-R is that it may crash the R console GUI if there is an error in your PDQ model.
    Known examples include:

7  Version 5.0.2 (2009)

Release 5.0.2 Build 070809.
  1. This unofficial release was withdrawn.
  2. If you jumped the gun and downloaded it from SourceForge, you should update to 5.0.3.

8  Version 5.0.1 (2009)

Major release 5.0.1, build 040209. PDQ 5.0 is a major release after 2 years (almost to the day). New features include:
  1. All development is now centrally organized at SourceForge.net (Phil Feller, Neil Gunther, Steve Jenkin)
  2. Improved organization of PDQ source files and simpler install scripts (Phil Feller)
  3. Multiserver queues as defined in Ch. 6 of the Perl::PDQ book (Neil Gunther)
  4. Integration with an installed R package (Phil Feller)
  5. Example PDQ-R models (Neil Gunther)
Some important benefits of these new features are: Now deprecated are:
  1. The Java implementation of PDQ (Peter Harding) in PDQ v4.2 has been retained as a stand-alone package.
  2. The PHP implementation of PDQ in PDQ v4.2 (Samuel Zallocco) has been retained as a stand-alone package.
  3. Neither of these implementations will be supported in future releases of PDQ. A Java implementation of PDQ based on JNI is currently in development.

9  Version 4.2.1 (2007)

Release 4.2.1 Build 031307. PDQ (Pretty Damn Quick) finally made it out the door as version 4.2 and is now available for immediate download. The PDQ models included in the /examples/ directory correspond to those discussed in each of my books, but PDQ is primarily associated with the Perl::PDQ book.
The main features of PDQ 4.2 are:
Complete installation instructions are available on the download page. Make sure you also read the various README files. Tom Becker (USA) and Christof Schmalenbach (Germany) have kindly provided separate installation notes for ActivePerl on Windows. This also indicates how international PDQ has become. :)

10  Version 4.0.1 (2007)

Major release 4.0.1, build 022507. Version 4.0 of PDQ (Pretty Damn Quick) is in the pipeline: it's been there for quite some time, actually (blush). The current hold up is related to getting both the Perl and the new Java version through the QA suite designed by Peter Harding. As soon as that is sorted out, we'll release it; probably in two pieces, keeping the Java PDQ separate initially. Also included will be updates to PyDQ (python) and a new PHP version.

11  C Language PDQ (1998, 2000)

A number of people have asked me for the C-language version of PDQ, especially those using my 1998/2000 book The Practical Performance Analyst. The C-code of those PDQ models can be found in the directory examples/ppa_1998/. The Perl scripts for the PDQ models discussed my 2005 book Analyzing Computer System Performance with Perl::PDQ, can now be found in the directory examples/ppdq_2005/. A detailed synopsis of the PDQ functions in Perl is presented in Chapter 6.

12  Online User Guide

The online PDQ Manual is now hyperlinked for easier navigation and supercedes both
  1. Appendix A in the The Practical Performance Analyst
  2. Appendix D in the Analyzing Computer System Performance with Perl::PDQ (2nd edn.).

13  Get Notified

The preferred way to keep abreast of PDQ changes is to join the Guerrilla Google Group. You can also ask questions there about using PDQ to solve performance problems.
Similarly, subscribe to the blog The Pith of Performance.
Alternatively, if you would like to be notified by email, please fill out the online form with your correct email address and select the heading Notify for PDQ updates. The same applies if you have changed your email address (e.g., changed employer).

File translated from TEX by TTH, version 3.81.
On 28 Jun 2016, 10:50.