PDQ (Pretty Damn Quick) Software Distribution

PDQ (Pretty Damn Quick) Software Distribution

This page was last edited on Apr 7, 2014

Current Distribution

Version:       6.1.1
Build:       011013
Languages:       C, Perl, Python, R, (Java, PHP)
Platforms:       Linux, Mac OS X, Windows, Solaris10
Developers:       Denny Chen, Phil Feller, Neil Gunther, Peter Harding, Paul Puglia

Contents

1  Release 6.1.1 Build 011013
    1.1  Building the Full PDQ Distribution
        1.1.1  Building Perl, Python, and R
        1.1.2  Perl5
        1.1.3  Python
        1.1.4  R
    1.2  Changes to PDQ Report Layout
        1.2.1  The Header
        1.2.2  Workload Parameters
        1.2.3  Resource Parameters
2  Release 6.0.1 Build 121512
        2.0.4  Changes to PDQ C-library for PDQ-R
        2.0.5  Special note on PDQ-R with Windows
        2.0.6  Beta Testing
3  Release 5.0.4 Build 030211
4  Release 5.0.3 Build 071209
5  Release 5.0.2 Build 070809
    5.1  Release 5.0.1 Build 040209
6  Release 4.2.1 Build 031307
7  Release 4.0.1 Build 022507
8  Downloading and Setup
    8.1  A Word About Windows
    8.2  Cygwin Shell for Windows
    8.3  PDQ-R on Windows and Cygwin
9  Generic Installation
    9.1  Generic Unix Environment
    9.2  Typical Install Procedure
    9.3  Manual Procedure
    9.4  Possible Unix Gotchas
10  Specific Installations
    10.1  Solaris Environment
    10.2  Linux Environment
    10.3  WINDOWS with ActiveState Perl
        10.3.1  Tom Becker's Notes
        10.3.2  Christof Schmalenbach's Notes
        10.3.3  Alex Podelko's Notes
11  Get Notified About Updates
12  Where's Waldo? (Anyone C-een him?)
13  User Guide

1  Release 6.1.1 Build 011013

This minor release (coming so soon after 6.0.1) was motivated by our desire to make more of the PDQ language extensions available on Windows; not just PDQ-R. This is particularly true for Perl, since that is the language used for all the PDQ models in the Perl::PDQ book.
We realized shortly after the release of 6.0.1 that the compilation procedure we developed for R could also be used as a template for re-packaging the other language extensions. In other words: packaging the code as a source tarball, building it with the language's native compiling facilities, and making each of them available to run under Windows. The specific details for building the full 6.1.1 distribution, along with each of the re-packaged extensions, are described below, beginning in Section 1.1.
The other minor change pertains to a reformatting of some sections of the output generated by the PDQ Report() function. See Section 1.2.
To acquire the full PDQ 6.1.1 distribution, download the zipped tarball from Sourceforge. The tarball now shows major, minor and patch release numbers. On *nix systems, make sure you have root privileges to avoid permissions problems.
If you only want to build PDQ specifically for one of the languages: Perl, Python or R—especially in a Microsoft Windows environment—see Section 1.1.1. Otherwise, continue reading and proceed to Section 1.1.
Whether you are a new or returning PDQ user, please join the Guerrilla google group and follow The Pith of Performance blog to keep abreast of all the latest developments.

1.1  Building the Full PDQ Distribution

In this release, two major changes have been made to the build process for the full distribution.
As in previous versions of PDQ, the build process for the full distribution is started by running make as root from the top-level directory. See the README file in the untar-ed download for full details.

1.1.1  Building Perl, Python, and R

Each of the three language extension packages shipped with PDQ (Perl5, Python, and R) have been reconfigured so they build using the "offical" compile method for their language (e.g. ExtUtils::MakeMakers for Perl.) This continues (and finishes) the process we started in 6.0.1 where we provided the R language extension as its own separate source package. By doing this, we've removed the need to build the full distribution in order to compile the extensions. All three extensions will now be available as separate, native source packages that can be built outside of the full distribution. And, like R, we'll be making each of them available in the PDQ download area on Sourceforge.
The following sections gives specifics on building and installing each of the three language extensions using their respective source packages.

1.1.2  Perl5

The PDQ Perl5 Extension has now been re-packaged as Perl source module that can be compiled using the ExtUtils::MakeMaker extension. While this greatly reduces the complexity of installing the extension on all platforms, on Windows, using the most current versions of Strawberry and ActiveState Perl, it provides a relatively staightforward means of making the extension available on that platform, without the need to also install a separate Unix/Linux-like environment such as Cygwin or MinGW.
The next two subsections provide details on how to install the Perl Extension for both Unix-like and Windows operating systems.
Unix, Linux, MacOS X:
  1. Download and unpack the perl tarball, pdq-6.1.1.tar.gz

  2. Change directory into the pdq-6.1.1

  3. Execute the following commands:
    1. perl Makefile.PL

    2. make

    3. make test—a PDQ report will be displayed if the execution of test.pl is successful

    4. make install



Windows:
The Perl PDQ module has been successfully built and tested using both Strawberry Perl and ActiveState Perl. Strawberry Perl ships with all the necessary compilers need for installing, and, if this is the version of Perl you're running, you can go immediately to step 1 below.
If you're running ActiveState Perl, you need to install two addition ActiveState PPMs before the build process can proceed. You can do this by using the following two commands: With both of these PPMs installed, you can now go to step 1 below.
Assuming all the prerequisites have already been met, you can build the Perl extension as follows:
  1. Download and unpack the perl tarball, pdq-6.1.1.tar.gz

  2. Open a CMD window as Administrator

  3. Change directory into the pdq-6.1.1

  4. Execute the following commands:
    1. Perl Makefile.PL

    2. dmake (Notice it's dmake, NOT make)

    3. dmake test—a PDQ report will be displayed if the execution of test.pl is successful

    4. dmake install



1.1.3  Python

Like the Perl extension, the Python extension has been reconfigured to build separately using the Python's Distutils. This should make the install process significantly easier for Linux,Unix and/or MacOS X users who are only interested in using this extension and have no interest in other parts of PDQ. At this point, there is no offical support for Windows. Technically, the extensions should build, but unlike Perl and R that ship with their on compiler environments for Windows, building Python on Windows requires using Visual C/C++ tools and no testing with this compiler has been done yet. We'd be interested in hearing about the experiences of users who try compiling on Windows.
The default procedure for building the Python extension is:
  1. Download and unpack the Python tarball, pdq-6.1.1.tar.gz

  2. Change directory to pdq-6.1.1

  3. The command: python setup.py install will build and install the package on most platforms, if it is run with root priviledges. On MacOS X, this can also be accomplished by prepending the sudo command as:
    sudo python setup.py install
    If you don't have root access on the install machine, a "local" compile can be included in your PYTHONPATH with the command:
    python setup.py install -user


  4. You can test whether or not the install is working correctly by running the test.py script located in the pdq-6.1.1 directory.

1.1.4  R

As of PDQ release 6.0.1, the PDQ-R extensions has been supplied as a separate source tar. It has been sucessfully installed on Linux, MacOS X and Windows. Building PDQ-R on Microsoft Windows is fairly simple process, even for those R Windows users who don't regularly build packages from sources. The steps are as follows:
  1. Download and install the R tools for Windows package from CRAN

  2. Download the PDQ-R source tarball pdq_6.1-1.tar.gz from the Sourceforge/Files page into a local directory e.g. C:\Users\Example\R-SRC
    Note the underscore in the tarball file name, which should not to be confused with the full PDQ distribution tarball.

  3. Start your version of R

  4. Use the command install.packages from the R console to install the package. The command will be something like this:
    install.packages("source_directory\\pdq_6.0-1.tar.gz",repos=NULL,type="source")


  5. For the source directory shown in the earlier example, the complete command would be:
    install.packages("C:\\Users\\Example\\R-SRC\\pdq_6.0-1.tar.gz",repos=NULL,type="source")


1.2  Changes to PDQ Report Layout

Changes have been made to the following sections in the output generated by the PDQ Report() function:
  1. Header section

  2. Workload Parameters section

  3. Resource Parameters section

1.2.1  The Header

The 6.1.1 Report header has been slimmed down to look like this:
 
                        PRETTY DAMN QUICK REPORT         
               ==========================================
               ***    of: Mon Feb 11 11:54:52 2013    ***
               ***   for: M/M/1 queue                 ***
               ***   Ver: PDQ Analyzer 6.1.1 011013   ***
               ==========================================

The surrounding borders have been simplified so as to consume less vertical space. The header width has been increased to accommodate the complete 3-level version numbering. Previously, only 2 levels were shown. The section title is now displayed in all caps to be consistent with other sections.

1.2.2  Workload Parameters

Single nodes, invoked with the CreateNode() function, are identified by the FCFS (first come first served) scheduling class while multiserver queues, invoked with the CreateMultiNode() function, are identified by the MSQ (multi server queue) scheduling class.
 
Node Sched Resource   Workload   Class     Demand
---- ----- --------   --------   -----     ------
  1  FCFS  Select     Calls      Open      0.5000
  3  MSQ   Claims     Calls      Open      3.3420
  7  MSQ   Policy     Calls      Open      9.2228

The Node column shows the number of servers available to each resource. That parameter now also reappears in the Resource section for easier cross-referencing.

1.2.3  Resource Parameters

Two new rows, identified by the following resource metrics:
  1. Capacity metric

  2. In service metric

have been added to this section. The new format looks like this:
 
Metric          Resource     Work              Value   Unit
------          --------     ----              -----   ----
Capacity        Policy       Calls                 7   Servers
Throughput      Policy       Calls            0.5833   Calls/Mins
In service      Policy       Calls            5.3800   Calls
Utilization     Policy       Calls           76.8567   Percent
Queue length    Policy       Calls            6.7826   Calls
Waiting line    Policy       Calls            1.4027   Calls
Waiting time    Policy       Calls            2.4046   Mins
Residence time  Policy       Calls           11.6274   Mins

The Capacity metric (1st row) indicates the number of servers available at that resource and is a repeat of the Node value in the Workload Parameters section. This can be used to check the intent of your PDQ model.
The In service metric (3rd row) refers to the average number of requests being served. It also indicates the total utilization of multi-server capacity.
In the above example, an average of 5.38 Policy servers out of a possible 7 servers are being utilized by the Calls workload. For a single server (i.e., a PDQ node with a Capacity Value of 1), the numerical value of the In service metric should be identical to the numerical value of the Utilization metric (4th row), except that the latter is expressed as a percentage (as it was in previous releases). Once again, this can be used as a cross check.

2  Release 6.0.1 Build 121512

If you are a new (or returning) user of PDQ, please join the Guerrilla google group and follow The Pith of Performance blog to keep abreast of all the latest developments.
Download the zipped tarball from Sourceforge. The tarball only shows major and minor 6.0 release numbers.
In a *nix shell, be sure to issue a sudo make command to avoid permissions problems.
The main purpose of this release is improved compatibility and stability between PDQ and the R statistical environment. Many of the PDQ models, previously found in the ../examples/ directory, can now also be accessed via the demo() command in the R-console.
After installing PDQ, issue the following commands in the R-console:
> library(pdq)
> demo(package="pdq")

and a list of available PDQ-R models will be shown. These PDQ-R scripts can be executed and studied entirely from within the R environment in the usual way. Use the help() function for an online introduction:
> help(pdq)

To get a listing of PDQ function calls, go to the bottom of the Help page in R and click on the Index link. Too see the R source code for the PDQ-R models, click on the Code demos link at the top of the PDQ listings page.
Testing was carried out using R version 2.15.2 (2012-10-26). Operationally, PDQ (of any flavor) should appear cosmetically the same as the release 5 version; no additional programming required. Further background information about this release can be found in the blog posts: PDQ 6.0 is On Its Way and PDQ 6.0 from a Developer Standpoint.

2.0.4  Changes to PDQ C-library for PDQ-R

The following changes to the PDQ C-library were made to better support PDQ-R: · Replaced printf, fprintf functions with macro PRINTF to support toggling between c-library functions and R API Rprintf() functions. This routes output to the R console instead of directly to stdout. The chief benefit of this that the R sink() function can now redirect the output of the PDQ-R Report() function.
· Added code as an #ifdef which toggles between standard c-library calloc() and free() and R API Calloc() and Free() functions. R API functions are used when compiling PDQ source files inside an R package. This allows PDQ to use R memory management APIs when running inside R.
· Added code as an #ifdef that toggles between standard c-library exit() and R API error() functions. This fixes the issue of PDQ-R causing the R console to end abruptly when an error condition is hit. Error messages are also redirected to the R console.
· Twenty six (26) demo scripts have been added to the PDQ-R package. A full listing of scripts can be seen by running demo(pdq) after the R package has been loaded into your environment using the command, library(pdq).
· With this release, PDQ-R is also being provided as an R source tar.gz. This tar ball is exactly the same code that the build process of the distribution produces. This should provide PDQ-R users who want to run package on Microsoft Windows a means of building and installing the package without the need to build the entire PDQ distribution.

2.0.5  Special note on PDQ-R with Windows

Building PDQ-R on Microsoft Windows is fairly simple process, even for those R Windows users who don't regularly build packages from sources. The steps to do are as follows:
· Download and install the R tools for Windows package from CRAN
· Download the PDQ-R source tar ball from the Sourceforge/Files page into a local directory (e.g. C:\Users\Example\R-SRC)
· Start your version of R
· Use the command install.packages from the R console to install the package. The command will be something like this:
install.packages("source_directory\\pdq_6.0-1.tar.gz",repos=NULL,type="source")
· For the source directory shown in the earlier example, the complete command would be:
install.packages("C:\\Users\\Example\\R-SRC\\pdq_6.0-1.tar.gz",repos=NULL,type="source")

2.0.6  Beta Testing

Thanks to Denny Chen and Ed Borasky for helping to test the beta release.

3  Release 5.0.4 Build 030211

  1. Fixed a bug that was identified on the GCaP forum. For models with multiple streams (workloads) on multiple nodes (queues), it was possible for the arrival rate of stream-B to be reported as exceeding the saturation throughtput of say, node-X, but using the (inverse) service demand of stream-A on node-X in the calculated comparison. In other words, the index cross-referencing was wrong. This problem only showed up at or above saturation thresholds, which is why it managed to escape previous detection in the QA test suite.

  2. Improved error messaging to show explicit stream and node names when saturation threshold is exceeded. For example, in the above case:
    ERROR in procedure 'canonical()': Arrival rate 34.560 for stream 'workB' exceeds saturation thruput 34.483 of node 'CPU' with demand 0.029


  3. Updated PDQ online manual synopsis for SetTUnit and SetWUnit. Must call CreateOpen or CreateClosed before calling SetWUnit or SetTUnit.

  4. Don't forget, you need to perform a sudo make in the top-level /pdq directory to ensure write permission for the various support directories.

4  Release 5.0.3 Build 071209

  1. This release ensures that all PDQ functions are available in each programming-language environment and fixes a bug in how mixed workloads are calculated, as well as a bug with displayed device utilizations in PDQ Report().

  2. PDQ can be built and installed by issuing a sudo make command in the top-level /pdq directory. The sudo is important to ensure you the right permissions to write into the various support directories. This is especially important for the R build to work.

  3. An as yet unresolved problem with PDQ-R is that it may crash the R console GUI if there is an error in your PDQ model.
    Known examples include:

5  Release 5.0.2 Build 070809

  1. This unofficial release was withdrawn.

  2. If you jumped the gun and downloaded it from SourceForge, you should update to 5.0.3.

5.1  Release 5.0.1 Build 040209

PDQ 5.0 is a major release after 2 years (almost to the day). New features include:
  1. All development is now centrally organized at SourceForge.net (Phil Feller, Neil Gunther, Steve Jenkin)

  2. Improved organization of PDQ source files and simpler install scripts (Phil Feller)

  3. Multiserver queues as defined in Ch. 6 of the Perl::PDQ book (Neil Gunther)

  4. Integration with an installed R package (Phil Feller)

  5. Example PDQ-R models (Neil Gunther)

Some important benefits of these new features are: Now deprecated are:
  1. The Java implementation of PDQ (Peter Harding) in PDQ v4.2 has been retained as a stand-alone package.

  2. The PHP implementation of PDQ in PDQ v4.2 (Samuel Zallocco) has been retained as a stand-alone package.

  3. Neither of these implementations will be supported in future releases of PDQ. A Java implementation of PDQ based on JNI is currently in development.

6  Release 4.2.1 Build 031307

PDQ (Pretty Damn Quick) finally made it out the door as version 4.2 and is now available for immediate download. The PDQ models included in the /examples/ directory correspond to those discussed in each of my books, but PDQ is primarily associated with the Perl::PDQ book.
The main features of PDQ 4.2 are:
Complete installation instructions are available on the download page. Make sure you also read the various README files. Tom Becker (USA) and Christof Schmalenbach (Germany) have kindly provided separate installation notes for ActivePerl on Windows. This also indicates how international PDQ has become. :)

7  Release 4.0.1 Build 022507

Version 4.0 of PDQ (Pretty Damn Quick) is in the pipeline: it's been there for quite some time, actually (blush). The current hold up is related to getting both the Perl and the new Java version through the QA suite designed by Peter Harding. As soon as that is sorted out, we'll release it; probably in two pieces, keeping the Java PDQ separate initially. Also included will be updates to PyDQ (python) and a new PHP version.

8  Downloading and Setup

Download the PDQ tarball from SourceForge.

8.1  A Word About Windows

The following information will becoming superceded since the PDQ 6.0 release. These notes are being retained in case they are still helpful for some users.
Sections 9 and 10 below discuss how to install PDQ in different OS environments. Many people are running some type of Windows environment. Windows does not provide a Perl interpreter by default so, you will need to install something like ActiveState Perl. See Section 10.3. You will also need to have access to a C compiler to compile the PDQ library, no matter which language you intend to use for your PDQ models. There are many compilers and IDE products available for that. The only thing to keep in mind is, that PDQ is not supported under any of these products.

8.2  Cygwin Shell for Windows

Prior to Mac OS X, PDQ was developed on Windows under Cygwin. Cygwin provides a Linux or UNIX style bash shell environment on top of Microsoft Windows.
Cygwin is not an emulation, in the sense of requiring a separate interpreter. All commands are exectutable .exe files and run directly under the Windows OS. Of course, you can also write integrated windows code using Windows .dll's or Tcl, etc. The Cygwin installer is capable of easily providing Perl, Python, gcc, etc., all in a self-contained environment that is accessed by a terminal window (similar to a Windows Commander console). This makes using PDQ under Windows much easier and cleaner than almost any other solution. Try it, you might like it.
You might need to customize your bashrc file so that it has the correct paths set up as vyrwin enviroment globals. Here is the .bashrc I used to run under cygwin:
export Perl5LIB=${HOME}/lib/perl5/site_perl/5.8/cygwin/auto/pdq

# User specific aliases and functions
alias ls="ls -lF --time-style=long-iso"
alias more='less'
alias c='clear'
alias h='history | more'
alias cygv='cygcheck -s | grep "DLL version: "'

# Source global definitions
if [ -f /etc/bashrc ]; then
      . /etc/bashrc
fi

# shell prompt
PS1="[\h:\w]% "

If, in addition, you have a .bash_profile file, it will automatically source your .bashrc each time you login. Here is the .bash_profile I used:
# ~/.bash_profile: executed by bash for login shells.

if [ -e /etc/bash.bashrc ] ; then
  source /etc/bash.bashrc
fi

if [ -e ~/.bashrc ] ; then
  source ~/.bashrc
fi

# Set PATH so it includes user's private bin if it exists
# if [ -d ~/bin ] ; then
#   PATH="~/bin:${PATH}"
# fi

# Set MANPATH so it includes users' private man if it exists
# if [ -d ~/man ]; then
#   MANPATH="~/man:${MANPATH}"
# fi

# Set INFOPATH so it includes users' private info if it exists
# if [ -d ~/info ]; then
#   INFOPATH="~/info:${INFOPATH}"
# fi

If you have any other experiences or suggestions, please share them so we can let others know.

8.3  PDQ-R on Windows and Cygwin

PDQ-R is not supported on the Windows version of R, at this time.
Apparently, the R package is not included in the official Cygwin distribution, but Cygwin Ports provides it separately. You can download it directly from the sourceforge repository.

9  Generic Installation

After downloading the tarball pdq.tar.gz, you should unzip it: gunzip pdq.tar.gz to produce the file named pdq.tar. Then you untar it: tar -xvf pdq.tar to produce the directory named pdq/ (or similar).
Once you are successful (or while waiting for your sys admin to set it up) you might care to review the basic concepts behind PDQ.
The following sections assume you are using a Unix or Linux-type environment. Otherwise, your results may vary. In particular, if you are using ActiveState Perl under a Windows O/S, see Section 10.3.

9.1  Generic Unix Environment

The Perl interpreter needed to run Perl-PDQ is available on a wide variety of platforms, and is already installed on most Unix and Linux platforms.

9.2  Typical Install Procedure

PDQ can be built and installed by issuing a sudo make command in the top-level /pdq directory.

9.3  Manual Procedure

Alternatively, it may be built manually by as follows:
  1. Change to the /pdq subdirectory (cd pdq)

  2. Change to the perl5 directory: cd perl5

  3. Run the setup script: ./setup.sh

  4. Go back up to the pdq directory: cd ..

Now, you should be able to run the code examples provided.
In case there is a problem compiling, perform the following diagnostic steps: Example Makefiles are already provided in that directory for the common operating systems. In some cases, you may need the help of a system administrator to get the files to compile correctly.
The basic goal is to generate a correct libpdq.a archive because it is used by everything.

9.4  Possible Unix Gotchas

The following items are typical of some local installation challenges that can prevent Perl PDQ models from executing.
See the README file in the Perl5/ directory of this distribution for more details about installation setup and debugging.

10  Specific Installations

10.1  Solaris Environment

The following notes from Stefan Parvu (Finland) refer to Solaris 11 on x86 platforms.
Typical issues that you will likely face are:
Stefan addressed these questions on 1/9/08 11:25 PM as follows.
1. Check pre-requisites
cc compiler:

$ which cc
/usr/bin/cc
$ cc -V 
cc: Sun C 5.9 SunOS_i386 Patch 124868-01 2007/07/12
usage: cc [ options] files.  Use 'cc -flags' for details

$ which make
/usr/bin/make

$ which dmake
/usr/bin/dmake


2. All delivered files are not proper under UNIX. Need to change the 
EOL character using dos2unix

Example, setup.sh:

#!/bin/sh^M
#^M
#  $Id: setup.sh,v 1.7 2006/04/08 23:56:15 zyx Exp $^M
#^M
^M
make clean^M
^M
ar xv ../lib/libpdq.a^M
^M
# swig -perl5 pdq.i^M
^M
perl Makefile.PL^M
^M
make install^M
^M

*** Perl5 Directory ***
$ for f in $(find . -type f); do print "Processing: $f"; dos2unix $f $f; done
Processing: ./.cvsignore
Processing: ./pdq_wrap_20070102.c
Processing: ./setup.sh
Processing: ./pdq.pm
Processing: ./pdq.i
Processing: ./pdq_wrap_old.c
Processing: ./pdq_wrap_x.c
Processing: ./Makefile.old
Processing: ./q
Processing: ./pdq_wrap.c
Processing: ./Makefile.PL
Processing: ./._test.out
dos2unix: ./._test.out not writable.  Permission denied.
Processing: ./README
Processing: ./test.pl
Processing: ./test.out

$ chmod 755 setup.sh


*** LIB Directory ***

$ pwd
/export/home/sparvu/workspace/PDQ/pdq42/lib
$ ls -lrt
total 195
-rwxr-xr-x   1 sparvu   eng          394 Feb 23  2004 debug.h
-rwxr-xr-x   1 sparvu   eng         3721 Oct  5  2004 MVA_Solve.c
-rwxr-xr-x   1 sparvu   eng         7515 Oct 30  2004 MVA_Approx.c
-rwxr-xr-x   1 sparvu   eng          634 Nov 19  2004 Makefile
-rwxr-xr-x   1 sparvu   eng         4515 May 10  2006 PDQ_Exact.c
-rwxr-xr-x   1 sparvu   eng         3366 May 13  2006 MVA_Canon.c
-rw-r--r--   1 sparvu   eng         1284 Jan  2  2007 PDQ_Global.h
-rw-r--r--   1 sparvu   eng         1302 Jan  2  2007 PDQ_Globals.c
-rw-r--r--   1 sparvu   eng        13166 Jan  2  2007 PDQ_Build.c
-rw-r--r--   1 sparvu   eng        15826 Jan 10  2007 PDQ_Utils.c
-rw-r--r--   1 sparvu   eng        18062 Jan 10  2007 PDQ_Report.c
-rwxr-xr-x   1 sparvu   eng         7259 Feb  9  2007 PDQ_Lib_copy.h
-rw-r--r--   1 sparvu   eng         7581 Feb 28  2007 PDQ_Lib.h
-rw-r--r--   1 sparvu   eng          642 Feb 28  2007 Makefile.cygwin
-rw-r--r--   1 sparvu   eng          497 Feb 28  2007 Makefile.solaris
-rw-r--r--   1 sparvu   eng          525 Feb 28  2007 Makefile.gcc
-rwxr-xr-x   1 sparvu   eng          573 Mar 14  2007 Makefile.macosx

Processing: ./Makefile.macosx
Processing: ./PDQ_Report.c
Processing: ./PDQ_Lib.h
Processing: ./PDQ_Lib_copy.h
Processing: ./PDQ_Exact.c
Processing: ./Makefile
Processing: ./Makefile.gcc
Processing: ./._Makefile.macosx
dos2unix: ./._Makefile.macosx not writable.  Permission denied.
Processing: ./MVA_Approx.c
Processing: ./MVA_Canon.c
Processing: ./PDQ_Globals.c
Processing: ./Makefile.cygwin
Processing: ./PDQ_Utils.c
Processing: ./PDQ_Global.h
Processing: ./PDQ_Build.c
Processing: ./Makefile.solaris
Processing: ./MVA_Solve.c
Processing: ./debug.h



3. libpdq.a 

Do we really need need the static library or not ?
Looks like {\tt setup.sh} for perl5 directory requires to have a {\tt libpdq.a} otherwise it won't compile clean. 
But read on ...

sparvu@nereid>make
cc -ggdb  -c  MVA_Approx.c
cc: Warning: illegal option -db
"MVA_Approx.c", line 41: warning: implicit function declaration: debug
"MVA_Approx.c", line 44: warning: implicit function declaration: errmsg
"MVA_Approx.c", line 54: warning: implicit function declaration: resets
cc -ggdb  -c  MVA_Canon.c
cc: Warning: illegal option -db
"MVA_Canon.c", line 39: warning: implicit function declaration: debug
"MVA_Canon.c", line 55: warning: implicit function declaration: errmsg
"MVA_Canon.c", line 95: warning: implicit function declaration: typetostr
"MVA_Canon.c", line 108: warning: implicit function declaration: getjob_name
cc -ggdb  -c  MVA_Solve.c
cc: Warning: illegal option -db
"MVA_Solve.c", line 33: warning: implicit function declaration: debug
"MVA_Solve.c", line 42: warning: implicit function declaration: typetostr
"MVA_Solve.c", line 47: warning: implicit function declaration: errmsg
"MVA_Solve.c", line 53: warning: implicit function declaration: exact
"MVA_Solve.c", line 72: warning: implicit function declaration: approx
"MVA_Solve.c", line 88: warning: implicit function declaration: canonical
"MVA_Solve.c", line 124: warning: implicit function declaration: getjob_name
cc -ggdb  -c  PDQ_Build.c
cc: Warning: illegal option -db
"PDQ_Build.c", line 62: warning: implicit function declaration: debug
"PDQ_Build.c", line 65: warning: implicit function declaration: resets
"PDQ_Build.c", line 67: warning: implicit function declaration: errmsg
"PDQ_Build.c", line 187: warning: implicit function declaration: close
"PDQ_Build.c", line 208: warning: implicit function declaration: typetostr
"PDQ_Build.c", line 364: warning: implicit function declaration: getnode_index
"PDQ_Build.c", line 365: warning: implicit function declaration: getjob_index
cc -ggdb  -c  PDQ_Exact.c
cc: Warning: illegal option -db
"PDQ_Exact.c", line 55: warning: implicit function declaration: errmsg
cc -ggdb  -c  PDQ_Globals.c
cc: Warning: illegal option -db
cc -ggdb  -c  PDQ_Report.c
cc: Warning: illegal option -db
"PDQ_Report.c", line 76: warning: implicit function declaration: resets
"PDQ_Report.c", line 87: warning: implicit function declaration: errmsg
"PDQ_Report.c", line 187: warning: implicit function declaration: typetostr
"PDQ_Report.c", line 239: warning: implicit function declaration: debug
"PDQ_Report.c", line 309: warning: implicit function declaration: getjob_name
cc -ggdb  -c  PDQ_Utils.c
cc: Warning: illegal option -db
"PDQ_Utils.c", line 85: warning: implicit function declaration: getjob_index
ar -rv libpdq.a MVA_Approx.o  MVA_Canon.o  MVA_Solve.o  PDQ_Build.o  PDQ_Exact.o  PDQ_Globals.o  PDQ_Report.o  PDQ_Utils.o
a - MVA_Approx.o
a - MVA_Canon.o
a - MVA_Solve.o
a - PDQ_Build.o
a - PDQ_Exact.o
a - PDQ_Globals.o
a - PDQ_Report.o
a - PDQ_Utils.o
ar: creating libpdq.a
ar: writing libpdq.a
# -ranlib -sc libpdq.a      # MacOS X
ranlib libpdq.a          # Cygwin


This creates a static library, using the cc compiler. However here I think we should use make -f Makefile.solaris
in order to get the shared library compiled. 


4. perl5 

$ ./setup.sh
make: Fatal error: Don't know how to make target `clean'
x - MVA_Approx.o
x - MVA_Canon.o
x - MVA_Solve.o
x - PDQ_Build.o
x - PDQ_Exact.o
x - PDQ_Globals.o
x - PDQ_Report.o
x - PDQ_Utils.o
Writing Makefile for pdq
cp pdq.pm blib/lib/pdq.pm
cc -c    -D_LARGEFILE_SOURCE -D_FILE_OFFSET_BITS=64 -D_TS_ERRNO -xO3 -xspace -xildoff    -DVERSION=\"\"  -DXS_VERSION=\"\" -KPIC "-I/usr/perl5/5.8.4/lib/i86pc-solaris-64int/CORE"   pdq_wrap.c
Running Mkbootstrap for pdq ()
chmod 644 pdq.bs
rm -f blib/arch/auto/pdq/pdq.so
LD_RUN_PATH="" cc  -G MVA_Approx.o MVA_Canon.o MVA_Solve.o PDQ_Build.o PDQ_Exact.o PDQ_Globals.o PDQ_Report.o PDQ_Utils.o pdq_wrap.o  -o blib/arch/auto/pdq/pdq.so      
chmod 755 blib/arch/auto/pdq/pdq.so
cp pdq.bs blib/arch/auto/pdq/pdq.bs
chmod 644 blib/arch/auto/pdq/pdq.bs
Installing /export/home/sparvu/lib/site_perl/5.8.4/i86pc-solaris-64int/auto/pdq/pdq.so
Files found in blib/arch: installing files in blib/lib into architecture dependent library tree
Writing /export/home/sparvu///lib/site_perl/5.8.4/i86pc-solaris-64int/auto/pdq/.packlist
Appending installation info to /export/home/sparvu///lib/i86pc-solaris-64int/perllocal.pod


If we try to compile only the .so then STEP 4 fails, because it complains it can't find the static lib libpdq.a - Any ideas ?


5. Perl libraries. 
Even as root ./setup.sh wont copy the libraries to the correct place. 

# ./setup.sh 
make: Fatal error: Don't know how to make target `clean'
x - MVA_Approx.o
x - MVA_Canon.o
x - MVA_Solve.o
x - PDQ_Build.o
x - PDQ_Exact.o
x - PDQ_Globals.o
x - PDQ_Report.o
x - PDQ_Utils.o
Writing Makefile for pdq
cp pdq.pm blib/lib/pdq.pm
cc -c    -D_LARGEFILE_SOURCE -D_FILE_OFFSET_BITS=64 -D_TS_ERRNO -xO3 -xspace -xildoff    -DVERSION=\"\"  -DXS_VERSION=\"\" -KPIC "-I/usr/perl5/5.8.4/lib/i86pc-solaris-64int/CORE"   pdq_wrap.c
Running Mkbootstrap for pdq ()
chmod 644 pdq.bs
rm -f blib/arch/auto/pdq/pdq.so
LD_RUN_PATH="" cc  -G MVA_Approx.o MVA_Canon.o MVA_Solve.o PDQ_Build.o PDQ_Exact.o PDQ_Globals.o PDQ_Report.o PDQ_Utils.o pdq_wrap.o  -o blib/arch/auto/pdq/pdq.so      
chmod 755 blib/arch/auto/pdq/pdq.so
cp pdq.bs blib/arch/auto/pdq/pdq.bs
chmod 644 blib/arch/auto/pdq/pdq.bs
Installing /lib/site_perl/5.8.4/i86pc-solaris-64int/auto/pdq/pdq.so
Files found in blib/arch: installing files in blib/lib into architecture dependent library tree
Writing ////lib/site_perl/5.8.4/i86pc-solaris-64int/auto/pdq/.packlist
Appending installation info to ////lib/i86pc-solaris-64int/perllocal.pod

10.2  Linux Environment

There shouldn't be many issues using PDQ under Linux, however I did receive the following email From Rodrigo Campos (Brazil) concerning SELinux (not Swedish Linux, security enabled Linux).
Wednesday, December 19, 2007 at 21:32:59:
Just to let you know that when you try to run any Perl::PDQ script under a
SELinux enabled Linux distribution, you'll get the following error
(example):

[rcampos@localhost pdq_models]$ ./passport.pl
Can't load '/home/rcampos/lib/perl5/site_perl/5.8.8/i386-linux-thread-multi//auto/pdq/pdq.so' 
for module pdq: /home/rcampos/lib/perl5/site_perl/5.8.8/i386-linux-thread-multi//auto/pdq/pdq.so: 
cannot restore segment prot after reloc: Permission denied at 
/usr/lib/perl5/5.8.8/i386-linux-thread-multi/DynaLoader.pm line 230.
 at ./passport.pl line 4
Compilation failed in require at ./passport.pl line 4.
BEGIN failed--compilation aborted at ./passport.pl line 4.


In order to solve this problem you need to change the security context of
the pdq.so shared object by using the following command:

[rcampos@localhost ~]$ sudo chcon -t texrel_shlib_t /home/rcampos/lib/perl5/site_perl/5.8.8/i386-linux-thread-multi/auto/pdq/pdq.*

This information may be useful to others as many linux distros are coming
with SELinux enforced by default (in my case I use CentOS).

10.3  WINDOWS with ActiveState Perl

I have now received two sets of instructions for installing ActivePerl by:
  1. Tom Becker - See Section 10.3.1

  2. Christof Schmalenbach - See Section 10.3.2

I don't know anything about ActivePerl (and I don't want to know), but I am very grateful to these users for providing their installation instructions which may be helpful to other ActivePerl users.

10.3.1  Tom Becker's Notes

The following extensive notes were kindly provided by Tom Becker who undertook a Herculean effort to get Perl::PDQ working under ActiveState Perl on Windows XP. Reading this makes realize why I use cygwin on my Windows laptop (sonyXPress). But if you must use Active Perl, this is the place to start. Thanks Tom!
This is my experience getting Perl::PDQ running in Windows XP. I have no C
skills, no UNIX skills, very limited Perl skills and am a PC user, not a
developer. All my information was gathered via Google and I hobbled together
all the required bits and pieces. All software used were free downloads.
Hopefully someone can drastically streamline my steps described below. 

ENVIRONMENT: Windows XP Pro with SP1, Pentium 4 CPU, ActiveState Perl 5.8.3
(mswin32-x86-multi-thread) binary build 809 with 8 patches.

INSTALLATION:
	1.	Installed the PDQ_Perl package as recommended. My main
working folder was D:\pdq_perl\perl5\pdq\perl5. A lot of work was done in a
DOS window.

	2.	Downloaded NMAKE15 from Microsoft. This is version 1.5 of
nmake (the equivalent of the UNIX make command). Unzip this into the
..\perl5 folder referenced above. It will add nmake.exe and nmake.err. See
//download.microsoft.com/download/vc15/Patch/1.52/W95/EN-US/Nmake15.exe.

	3.	Run Perl makefile.pl.

	4.	ActiveState Perl calls for the cl compiler (Microsoft's C++
compiler). I downloaded a free copy of the Microsoft Visual C++ Toolkit
2003. Installed it and ran VCVARS32.BAT to set up the environmental
variables of LIB, PATH and INCLUDE.

         5.  Downloaded the Windows Server 2003 SP1 Platform SDK Full
Download and picked out windows.h and all associated include header files
and put them in the \Perl\lib\CORE folder. I did not install this product.
The full download is 400MB and has 16 separate 25MB files. A short cut is to
download two cab files from "Windows Server 2003 SP1 Platform SDK Full
Download": PSDK-FULL.6.cab and PSDK-FULL.7.cab. (Also get PSDK-FULL.15.cab
for a later step below.) The contents can be extracted with WINZIP. Extract
from the two files above: PSDK-SDK_Core_BLD-commom.0.cab,
PSDK-SDK_Core_BLD-commom.1.cab, PSDK-SDK_Core_BLD-commom.2.cab. Extract all
the contents of the above three files to one folder. Then sort by file name.
Copy and rename 27 header files to \perl\lib\core folder. Example:
windows_h.95CE7D7B_A68D_11D2_A852_00C04FC2A854 renamed to windows.h. Extract
the following:

BaseTsd.h
Guiddef.h
Imm.h
Mcx.h
PopPack.h
PshPack1.h
PshPack2.h
PshPack4.h
PshPack8.h
Reason.h
specstrings.h
StrAlign.h
Tvout.h
winbase.h
WinCon.h
WinDef.h
windows.h
WinError.h
WinGDI.h
WinNetWk.h
WinNls.h
WinNT.h
WinReg.h
WinSock.h
WinSvc.h
WinUser.h
WinVer.h

	6.	Modified  d:\pdq_perl\perl5\pdq\lib\debug.h to remove the
three periods (...) in this statement:
	 	#define g_debugf(fmt, args...) \
			fprintf(stderr, "%s:%d " fmt, __FILE__, __LINE__,
##args)

	7.	Compiled all the c files in d:\pdq_perl\perl5\pdq\lib and
moved the .obj files to ..\perl5. There will be a warning message compiling
PDQ_Utils due to the modification to debug.h above.

	8.	Extract from PSDK-FULL.15.cab this file:
PSDK-SDK_MAC_BLD_X86-common.o.cab. From this file extract ODBC32.lib and
ODBCCP32.lib and move to ..\perl5.

	9.	Copy from the folder created in step 5 above (the extraction
of PSDK-SDK_Core_BLD-commom.0.cab,   PSDK-SDK_Core_BLD-commom.1.cab,
PSDK-SDK_Core_BLD-commom.2.cab) and move the following to ..\perl5:

	 	AdvAPI32.Lib
		ComDlg32.Lib
		Gdi32.Lib
		Mpr.Lib
		NetAPI32.Lib
		Ole32.Lib
		OleAut32.Lib
		Shell32.Lib
		User32.Lib
		Uuid.Lib
		Version.Lib
		WinMM.Lib
		WinSpool.Lib
		WSock32.Lib
		
	10.	Downloaded and installed Microsoft .NET Framework SDK
Version 1.1 to get msvcrt.lib even though there was a copy of it in the
Windows Server 2003 SP1 Platform SDK ; it was for AMD64 and would not link.
After installation msvcrt.lib is found in \Program Files\Microsoft visual
Studio .NET\VC7\lib folder. Moved it to ...\perl5 with the other libs from
above. 

	11.	Ran nmake.

	12.	Ran nmake install.

	13.	Ran the test.pl PDQ file to a successful completion.

RECOMMENDATIONS: (For the next pioneer):
1. Investigate why ActiveState Perl call for the cl.exe compiler (MS C++) in
the makefile. Perhaps a simpler compiler can be used.

2. Find out where all the Activestate Perl libs are that are displayed with
the perl -V command. It lists all the libs I had to search for, but were not
on my PC. So how does Activestate Perl use them?

3. Try to install the entire Windows Server 2003 SP1 Platform SDK FULL
DOWNLOAD. Maybe I overlooked msvcrt.lib for x86. However, I did inspect the
x86.msi file and it did reference the AMD64 file, which I tried but was
rejected by the link program. It will also simplify extracting all the
header files. But it is a 400MB download.

4. Get Microsoft to allow the redistribution of the headers and libs, rather
than forcing a download of the products I needed. 

10.3.2  Christof Schmalenbach's Notes

I (CS) use ActivePerl, but not cygwin. After some experiments I found a way of installation of Perl::PDQ under win32 without extensive use of MS Tools. Here is my approach:
1. Download nmake as descriped by Tom Becker
2. Download MinGW - Minimalist GNU for Windows (www.mingw.org) and install it
3. Open a DOS Shell  and extend your path env variable so that mingw\bin is in front of perl.
set path=d:\programs\MinGW\mingw32\bin;d:\programs\MinGW\bin;D:\programs\perl\bin in my environment
4. change to the pdq\lib directory
5. use gcc (from mingw) to compile pdq c files to object files (for example MVA_Approx.o..)
6. copy or move the object files to the ..\pdq\perl5 directory
7. call perl MakeFile.PL
8. call nmake MakeFile
9. after a while this creates ..pdq\perl5\blib\arch\auto\pdq\pdq.dll

With this dll I can use Perl::PDQ without problems.

IMPORTANT:
If mingw\bin is not in the path, perl MakeFile.PL will create cl.exe calls (as Tom remarked). 
But if Mingw\bin is in the path, the gcc compiler is used instead of MS.
So in general there is not a lot of difference between linux/cygwin and win32 for c-calls from perl.

BTW, there is a document (perlxstut.html) in the activeperl distribution, 
which explains the general approach for Perl extensions. 

10.3.3  Alex Podelko's Notes

Have built perl pdq.dll for Windows ... a real headache (even having VisualStudio already installed). Just want to share the last problem I got, not mentioned in the previous install experiences. After I built pdq.dll and ran test.pl I got the following error:
C:\Projects\pdq42\perl5>perl test.pl
Can't load './pdq.dll' for module pdq: load_file:A dynamic link library (DLL) in
itialization routine failed at C:/Perl/lib/DynaLoader.pm line 230.
 at test.pl line 6
Compilation failed in require at test.pl line 6.
BEGIN failed--compilation aborted at test.pl line 6.

After a couple of days I found the solution:
The following command solved the problem:
mt -manifest pdq.dll.manifest -outputresource:pdq.dll;2
The DLL is available for download. I did it with ActivePerl 5.8.8 and VC80 ... not sure how portable it is (I see dependencies from perl58.dll and msvcr80.dll).

11  Get Notified About Updates

If you would like to be notified by email about future PDQ updates, please fill out the online form with your correct email address and select the heading Notify for PDQ updates. The same applies if you have changed your email address (e.g., changed employer).

12  Where's Waldo? (Anyone C-een him?)

A number of people have asked me for the C-language version of PDQ, especially those using my 1998/2000 book The Practical Performance Analyst. The C-code of those PDQ models can be found in the directory examples/ppa_1998/. The Perl scripts for the PDQ models discussed my 2005 book Analyzing Computer System Performance with Perl::PDQ, can now be found in the directory examples/ppdq_2005/. A detailed synopsis of the PDQ functions in Perl is presented in Chapter 6.

13  User Guide

The online PDQ Manual for the C version of PDQ is now hyperlinked for easier navigation and supercedes Appendix A in the The Practical Performance Analyst.



File translated from TEX by TTH, version 3.38.
On 7 Apr 2014, 10:42.