This minor release (coming so soon after 6.0.1) was motivated by our desire to
make more of the PDQ language extensions available on Windows; not just PDQ-R.
This is particularly true for Perl, since that is the language used for all the PDQ models
in the Perl::PDQ book.
We realized shortly after the release of 6.0.1 that the compilation procedure we
developed for R could also be used as a template for re-packaging the other
language extensions. In other words: packaging the code as a source tarball, building it with
the language's native compiling facilities, and making each of them available to run under
Windows. The specific details for building the full 6.1.1 distribution, along with
each of the re-packaged extensions, are described below, beginning in Section 1.1.
The other minor change pertains to a reformatting of some sections of the output generated by the
PDQ Report() function.
See Section 1.2.
To acquire the full PDQ 6.1.1 distribution, download the zipped tarball from
Sourceforge.
The tarball now shows major, minor and patch release numbers.
On *nix systems, make sure you have root privileges to avoid permissions problems.
If you only want to build PDQ specifically for one of the languages: Perl, Python or R—especially
in a
Microsoft Windows environment—see Section 1.1.1.
Otherwise, continue reading and proceed to Section 1.1.
Whether you are a new or returning PDQ user, please join the
Guerrilla google group
and follow The Pith of Performance blog
to keep abreast of all the latest developments.
In this release, two major changes have been made to the build process for the full distribution.
The build process runs using only Gnumake without resorting to any of the
shell scripts that where a major part of the process in prior verions. This
change was made to make the build process more uniform across all of PDQ's
components. However, it has only been fully tested on Linux and MacOS X
platforms. Users trying to build on Solaris or any of the Unix-like
environments on Windows (Cygwin or MinGW) might run into issues. If you are one
of those users, please notify us via the Google Gurillia Group, and we'll try to
address it.
Unlike earily versions of PDQ, where the build process ignored build errors
while building the extensions, the 6.1.1 build process now aborts when any error
occurs. This means, if you're looking to build the full distribution or the
C-library, you'll need to have R, Perl5, and Python installed on your system.
Otherwise, the entire build will fail when it tries compiling the extension for
the missing language. We made this change to help debug build issues. We've been
getting a lot of support questions where users report that one of the extensions
isn't working, only to find that it failed to build without them realizing it.
As in previous versions of PDQ, the build process for the full distribution is started by
running make as root from the top-level directory. See the README file in the
untar-ed download for full details.
Each of the three language extension packages shipped with PDQ (Perl5, Python, and R) have been
reconfigured so they build using the "offical" compile method for their language
(e.g. ExtUtils::MakeMakers for Perl.) This continues (and finishes) the process
we started in 6.0.1 where we provided the R language extension as its own
separate source package. By doing this, we've removed the need to build the
full distribution in order to compile the extensions. All three extensions will
now be available as separate, native source packages that can be built outside
of the full distribution. And, like R, we'll be making each of them available in
the PDQ download area on
Sourceforge.
The following sections gives specifics on building and installing each of the
three language extensions using their respective source packages.
The PDQ Perl5 Extension has now been re-packaged as Perl source module that can
be compiled using the ExtUtils::MakeMaker extension. While this greatly reduces
the complexity of installing the extension on all platforms, on Windows, using
the most current versions of Strawberry and ActiveState Perl, it provides a
relatively staightforward means of making the extension available on that
platform, without the need to also install a separate Unix/Linux-like
environment such as Cygwin or MinGW.
The next two subsections provide details on how to install the Perl Extension
for both Unix-like and Windows operating systems.
Unix, Linux, MacOS X:
Download and unpack the perl tarball, pdq-6.1.1.tar.gz
Change directory into the pdq-6.1.1
Execute the following commands:
perl Makefile.PL
make
make test—a PDQ report will be displayed if the execution of test.pl is successful
make install
Windows:
The Perl PDQ module has been successfully built and tested using both Strawberry
Perl and ActiveState Perl. Strawberry Perl ships with all the necessary
compilers need for installing, and, if this is the version of Perl you're
running, you can go immediately to step 1 below.
If you're running ActiveState Perl, you need to install two addition
ActiveState PPMs before the build process can proceed. You can do this by using
the following two commands:
ppm install dmake
ppm install MinGW or ppm install MinGW64 for 64-bit platforms.
You need to be running ActiveState 5.16 build 1601 or greater.
With both of these PPMs installed, you can now go to step 1 below.
Assuming all the prerequisites have already been met, you can build the Perl extension as follows:
Download and unpack the perl tarball, pdq-6.1.1.tar.gz
Open a CMD window as Administrator
Change directory into the pdq-6.1.1
Execute the following commands:
Perl Makefile.PL
dmake (Notice it's dmake, NOT make)
dmake test—a PDQ report will be displayed if the execution of test.pl is successful
Like the Perl extension, the Python extension has been reconfigured to build
separately using the Python's Distutils. This should make the install process
significantly easier for Linux,Unix and/or MacOS X users who are only interested
in using this extension and have no interest in other parts of PDQ. At this
point, there is no offical support for Windows. Technically, the extensions
should build, but unlike Perl and R that ship with their on compiler
environments for Windows, building Python on Windows requires using Visual C/C++
tools and no testing with this compiler has been done yet. We'd be interested
in hearing about the experiences of users who try compiling on Windows.
The default procedure for building the Python extension is:
Download and unpack the Python tarball, pdq-6.1.1.tar.gz
Change directory to pdq-6.1.1
The command: python setup.py install
will build and install the package on most platforms, if it is run with root priviledges.
On MacOS X,
this can also be accomplished by prepending the sudo command as:
sudo python setup.py install
If you don't have root access on the install machine, a "local" compile
can be included in your PYTHONPATH with the command:
python setup.py install -user
You can test whether or not the install is working correctly by running the test.py script
located in the pdq-6.1.1 directory.
As of PDQ release 6.0.1, the PDQ-R extensions has been supplied as a separate source tar.
It has been sucessfully installed on Linux, MacOS X and Windows.
Building PDQ-R on Microsoft Windows is fairly simple process, even for
those R Windows users who don't regularly build packages from sources. The steps
are as follows:
Download and install the
R tools for Windows package from CRAN
Download the
PDQ-R source
tarball pdq_6.1-1.tar.gz from the Sourceforge/Files page into a local directory
e.g. C:\Users\Example\R-SRC Note the underscore in the tarball file name, which should not to be confused with the full PDQ distribution tarball.
Start your version of R
Use the command install.packages from the R console to install the package.
The command will be something like this:
The surrounding borders have been simplified so as to consume less vertical space.
The header width has been increased to accommodate the complete 3-level version numbering.
Previously, only 2 levels were shown. The section title is now displayed in all caps to be consistent with other sections.
Single nodes, invoked with the
CreateNode() function,
are identified by the FCFS (first come first served) scheduling class
while multiserver queues, invoked with the
CreateMultiNode() function,
are identified by the MSQ (multi server queue) scheduling class.
Node Sched Resource Workload Class Demand
---- ----- -------- -------- ----- ------
1 FCFS Select Calls Open 0.5000
3 MSQ Claims Calls Open 3.3420
7 MSQ Policy Calls Open 9.2228
The Node column shows the number of servers available to each resource.
That parameter now also reappears in the Resource section for easier cross-referencing.
Two new rows, identified by the following resource metrics:
Capacity metric
In service metric
have been added to this section. The new format looks like this:
Metric Resource Work Value Unit
------ -------- ---- ----- ----
Capacity Policy Calls 7 Servers
Throughput Policy Calls 0.5833 Calls/Mins
In service Policy Calls 5.3800 Calls
Utilization Policy Calls 76.8567 Percent
Queue length Policy Calls 6.7826 Calls
Waiting line Policy Calls 1.4027 Calls
Waiting time Policy Calls 2.4046 Mins
Residence time Policy Calls 11.6274 Mins
The Capacity metric (1st row) indicates the number of servers available at that resource
and is a repeat of the Node value in the Workload Parameters section.
This can be used to check the intent of your PDQ model.
The In service metric (3rd row) refers to the average number of requests being served. It also
indicates the total utilization of multi-server capacity.
In the above example, an average of 5.38 Policy servers out of a possible 7 servers are being utilized by the
Calls workload.
For a single server (i.e., a PDQ node with a Capacity Value of 1), the numerical value of the In service metric should be identical to the numerical value of the Utilization metric (4th row), except that the latter is expressed as a
percentage (as it was in previous releases). Once again, this can be used as a cross check.
If you are a new (or returning) user of PDQ, please join the
Guerrilla google group
and follow The Pith of Performance blog
to keep abreast of all the latest developments.
Download the zipped tarball from
Sourceforge.
The tarball only shows major and minor 6.0 release numbers.
In a *nix shell, be sure to issue a
sudo make command to avoid permissions problems.
The main purpose of this release is improved compatibility and stability between PDQ and the
R statistical environment.
Many of the PDQ models,
previously found in the ../examples/ directory, can now also be accessed via the demo()
command in the R-console.
After installing PDQ, issue the following commands in the R-console:
> library(pdq)
> demo(package="pdq")
and a list of available PDQ-R models will be shown.
These PDQ-R scripts can be executed and studied entirely from within the
R environment in the usual way. Use the help() function for an online introduction:
> help(pdq)
To get a listing of PDQ function calls, go to the bottom of the Help page in R and click on the Index link.
Too see the R source code for the PDQ-R models, click on the Code demos link at the top of the PDQ listings page.
Testing was carried out using R version 2.15.2 (2012-10-26).
Operationally, PDQ (of any flavor) should appear cosmetically the same as the release 5 version;
no additional programming required.
Further background information about this release can be found in the blog posts:
PDQ 6.0 is On Its Way and
PDQ 6.0 from a Developer Standpoint.
The following changes to the PDQ C-library were made to better support PDQ-R:
· Replaced printf, fprintf functions with macro PRINTF to support toggling between
c-library functions and R API Rprintf() functions. This routes output to the R
console instead of directly to stdout. The chief benefit of this that the R
sink() function can now redirect the output of the PDQ-R Report() function.
· Added code as an #ifdef which toggles between standard c-library calloc() and
free() and R API Calloc() and Free() functions. R API functions are used when
compiling PDQ source files inside an R package. This allows PDQ to use R memory
management APIs when running inside R.
· Added code as an #ifdef that toggles between standard c-library exit() and R API
error() functions. This fixes the issue of PDQ-R causing the R console to end
abruptly when an error condition is hit. Error messages are also redirected to
the R console.
· Twenty six (26) demo scripts have been added to the PDQ-R package. A full
listing of scripts can be seen by running demo(pdq) after the R package has been
loaded into your environment using the command, library(pdq).
· With this release, PDQ-R is also being provided as an R source tar.gz. This tar
ball is exactly the same code that the build process of the distribution
produces. This should provide PDQ-R users who want to run package on Microsoft
Windows a means of building and installing the package without the need to
build the entire PDQ distribution.
Building PDQ-R on Microsoft Windows is fairly simple process, even for those R Windows
users who don't regularly build packages from sources. The steps to do are as
follows:
· Download and install the R tools for Windows package from
CRAN
· Download the PDQ-R source tar ball from the
Sourceforge/Files
page into a local directory (e.g. C:\Users\Example\R-SRC)
· Start your version of R
· Use the command install.packages from the R console to install the package.
The command will be something like this: install.packages("source_directory\\pdq_6.0-1.tar.gz",repos=NULL,type="source")
· For the source directory shown in the earlier example, the complete command would be: install.packages("C:\\Users\\Example\\R-SRC\\pdq_6.0-1.tar.gz",repos=NULL,type="source")
Fixed a bug that was identified on the
GCaP forum.
For models with multiple streams (workloads) on multiple nodes (queues), it
was possible for the arrival rate of stream-B to be reported as exceeding
the saturation throughtput of say, node-X, but using the (inverse) service demand of
stream-A on node-X in the calculated comparison. In other words, the index cross-referencing was wrong.
This problem only showed up at or above saturation thresholds, which is why it managed
to escape previous detection in the QA test suite.
Improved error messaging to show explicit stream and node names when saturation threshold is exceeded.
For example, in the above case:
ERROR in procedure 'canonical()':
Arrival rate 34.560 for stream 'workB' exceeds saturation thruput
34.483 of node 'CPU' with demand 0.029
Updated PDQ online manual synopsis for
SetTUnit
and
SetWUnit.
Must call CreateOpen or CreateClosed before calling SetWUnit or SetTUnit.
Don't forget, you need to perform a sudo make in the top-level /pdq directory
to ensure write permission for the various support directories.
This release ensures that all PDQ functions are available in each
programming-language environment and fixes a bug in how mixed workloads are
calculated,
as well as a bug with displayed device utilizations in PDQ Report().
PDQ can be built and installed by issuing a sudo make
command in the top-level /pdq directory. The sudo is important
to ensure you the right permissions to write into the various support
directories. This is especially important for the
R build to work.
An as yet unresolved problem with PDQ-R
is that it may crash the R console GUI if there is an error in your PDQ model.
Known examples include:
utilization of a PDQ node exceeds 100%
reference is made to an unnamed or incorrectly named queue
Some important benefits of these new features are:
As well as centralized development,
SourceForge also provides a forum
page for Help and Discussions.
Due to various unforeseen circumstances (as always), the implementation of multiserver queues (MSQ)
did not occur until well after the Perl::PDQ book
was released in 2005.
As a consequence, the application of PDQ with MSQ to Erlang's queueing model (1917) and the extension to Jackson's theorem (1957)
did not appear in the book. However, you can find this information
in Section 3 of the online development notes
from 2007. Think of this material as a sneak preview into a portion of the next edition of the
Perl::PDQ book.
The integration with R is very significant because it does not have a qeueing network solver, but now
with PRQ-R you can:
import data using standard R commands
manipulate those data statistically
extract PDQ
input parameters, e.g.,
arrival rates, service times
set up a PDQ model within your R script
solve your PDQ model and display results using the
R plotting capability
compare those PDQ-R outputs with other imported R data
rinse and repeat
Now
deprecated are:
The Java implementation of PDQ (Peter Harding) in PDQ v4.2 has been retained as a stand-alone package.
The PHP implementation of PDQ in PDQ v4.2 (Samuel Zallocco) has been
retained as a stand-alone package.
Neither of these implementations will be supported in future releases of PDQ.
A Java implementation of PDQ based on JNI is currently in development.
PDQ (Pretty Damn Quick) finally made it out the door as version 4.2 and is now
available for immediate download. The PDQ models included in the /examples/
directory correspond to those discussed in each of my books, but PDQ is
primarily associated with the Perl::PDQ book.
The main features of PDQ 4.2 are:
Java version of PDQ contributed by Peter Harding in Australia
PHP version of PDQ contributed by Italian student Samuel Zallocco
Threaded-server model and errata corrections contributed by Neil Gunther
Better organization of the Perl and Python models
The Java and PHP packages are released stand-alone at this time
Complete installation instructions are available on the download page. Make sure
you also read the various README files. Tom Becker (USA) and Christof
Schmalenbach (Germany) have kindly provided separate installation notes for
ActivePerl on Windows. This also indicates how international PDQ has become. :)
Version 4.0 of PDQ (Pretty Damn Quick) is in the pipeline: it's been there for
quite some time, actually (blush). The current hold up is related to getting
both the Perl and the new Java version through the QA suite designed by Peter
Harding. As soon as that is sorted out, we'll release it; probably in two
pieces, keeping the Java PDQ separate initially. Also included will be updates
to PyDQ (python) and a new PHP version.
The following information will becoming superceded since the PDQ 6.0 release. These notes are being retained
in case they are still helpful for some users.
Sections 9 and 10 below discuss how to install PDQ in different OS environments.
Many people are running some type of Windows environment. Windows does not provide a Perl interpreter by default
so, you will need to install something like ActiveState Perl. See Section 10.3.
You will also need to have access to a C compiler to compile the PDQ library, no matter which language you intend
to use for your PDQ models. There are many compilers and IDE products available for that.
The only thing to keep in mind is, that PDQ is not supported under any of these products.
Prior to Mac OS X, PDQ was developed on Windows under Cygwin.
Cygwin provides a Linux or UNIX style bash shell
environment on top of Microsoft Windows.
Cygwin is not an emulation, in the sense of requiring a separate interpreter.
All commands are exectutable .exe files
and run directly under the Windows OS. Of course, you can also write integrated windows code using Windows
.dll's or Tcl, etc. The Cygwin installer is capable of easily providing Perl, Python, gcc, etc., all in a self-contained
environment that is accessed by a terminal window (similar to a Windows Commander console).
This makes using PDQ under Windows much easier and cleaner than almost any other solution.
Try it, you might like it.
You might need to customize your bashrc file so that it has the correct paths set up as vyrwin enviroment globals.
Here is the .bashrc I used to run under cygwin:
export Perl5LIB=${HOME}/lib/perl5/site_perl/5.8/cygwin/auto/pdq
# User specific aliases and functions
alias ls="ls -lF --time-style=long-iso"
alias more='less'
alias c='clear'
alias h='history | more'
alias cygv='cygcheck -s | grep "DLL version: "'
# Source global definitions
if [ -f /etc/bashrc ]; then
ÊÊ Ê Ê. /etc/bashrc
fi
# shell prompt
PS1="[\h:\w]% "
If, in addition, you have a .bash_profile file, it will automatically source your .bashrc each time you login.
Here is the .bash_profile I used:
# ~/.bash_profile: executed by bash for login shells.
if [ -e /etc/bash.bashrc ] ; then
ÊÊsource /etc/bash.bashrc
fi
if [ -e ~/.bashrc ] ; then
ÊÊsource ~/.bashrc
fi
# Set PATH so it includes user's private bin if it exists
# if [ -d ~/bin ] ; then
# Ê PATH="~/bin:${PATH}"
# fi
# Set MANPATH so it includes users' private man if it exists
# if [ -d ~/man ]; then
# Ê MANPATH="~/man:${MANPATH}"
# fi
# Set INFOPATH so it includes users' private info if it exists
# if [ -d ~/info ]; then
# Ê INFOPATH="~/info:${INFOPATH}"
# fi
If you have any other experiences or suggestions, please share them so we can let others know.
PDQ-R
is not supported on the
Windows version of R, at this time.
Apparently, the R package is
not included
in the official Cygwin distribution, but
Cygwin Ports provides it separately.
You can download it directly from the
sourceforge
repository.
After downloading the tarball pdq.tar.gz, you should
unzip it: gunzip pdq.tar.gz to produce the file named pdq.tar. Then you
untar it: tar -xvf pdq.tar to produce the directory named pdq/ (or similar).
Once you are successful (or while waiting for your sys admin to set it up)
you might care to review the
basic concepts
behind PDQ.
The following sections assume you are using a Unix or Linux-type environment. Otherwise, your
results may vary. In particular, if you are using ActiveState Perl under a Windows O/S, see
Section 10.3.
Alternatively, it may be built manually by as follows:
Change to the /pdq subdirectory (cd pdq)
Change to the perl5 directory: cd perl5
Run the setup script: ./setup.sh
Go back up to the pdq directory: cd ..
Now, you should be able to run the code examples provided.
In case there is a problem compiling, perform the following diagnostic steps:
Check the PDQ version. Run the command ./Getversion to confirm that you
have the same version number as above.
Go into the /lib subdirectory and make sure you are using the appropriate Makefile or
you may need to create your own---depending on your environment.
Example Makefiles are already provided in that directory for the common operating systems.
In some cases, you may need the help of a system administrator to get the files to compile correctly.
The basic goal is to generate a correct libpdq.a archive because it is used by everything.
The following notes from Stefan Parvu (Finland) refer to Solaris 11 on x86 platforms.
Typical issues that you will likely face are:
What compiler should be used: cc or gcc ?
Do you really need to use libpdq.a? (See steps 3,4)
Do you need to manually copy the Perl libraries into /usr/perl5?
Stefan addressed these questions on 1/9/08 11:25 PM as follows.
1. Check pre-requisites
cc compiler:
$ which cc
/usr/bin/cc
$ cc -V
cc: Sun C 5.9 SunOS_i386 Patch 124868-01 2007/07/12
usage: cc [ options] files. Use 'cc -flags' for details
$ which make
/usr/bin/make
$ which dmake
/usr/bin/dmake
2. All delivered files are not proper under UNIX. Need to change the
EOL character using dos2unix
Example, setup.sh:
#!/bin/sh^M
#^M
# $Id: setup.sh,v 1.7 2006/04/08 23:56:15 zyx Exp $^M
#^M
^M
make clean^M
^M
ar xv ../lib/libpdq.a^M
^M
# swig -perl5 pdq.i^M
^M
perl Makefile.PL^M
^M
make install^M
^M
*** Perl5 Directory ***
$ for f in $(find . -type f); do print "Processing: $f"; dos2unix $f $f; done
Processing: ./.cvsignore
Processing: ./pdq_wrap_20070102.c
Processing: ./setup.sh
Processing: ./pdq.pm
Processing: ./pdq.i
Processing: ./pdq_wrap_old.c
Processing: ./pdq_wrap_x.c
Processing: ./Makefile.old
Processing: ./q
Processing: ./pdq_wrap.c
Processing: ./Makefile.PL
Processing: ./._test.out
dos2unix: ./._test.out not writable. Permission denied.
Processing: ./README
Processing: ./test.pl
Processing: ./test.out
$ chmod 755 setup.sh
*** LIB Directory ***
$ pwd
/export/home/sparvu/workspace/PDQ/pdq42/lib
$ ls -lrt
total 195
-rwxr-xr-x 1 sparvu eng 394 Feb 23 2004 debug.h
-rwxr-xr-x 1 sparvu eng 3721 Oct 5 2004 MVA_Solve.c
-rwxr-xr-x 1 sparvu eng 7515 Oct 30 2004 MVA_Approx.c
-rwxr-xr-x 1 sparvu eng 634 Nov 19 2004 Makefile
-rwxr-xr-x 1 sparvu eng 4515 May 10 2006 PDQ_Exact.c
-rwxr-xr-x 1 sparvu eng 3366 May 13 2006 MVA_Canon.c
-rw-r--r-- 1 sparvu eng 1284 Jan 2 2007 PDQ_Global.h
-rw-r--r-- 1 sparvu eng 1302 Jan 2 2007 PDQ_Globals.c
-rw-r--r-- 1 sparvu eng 13166 Jan 2 2007 PDQ_Build.c
-rw-r--r-- 1 sparvu eng 15826 Jan 10 2007 PDQ_Utils.c
-rw-r--r-- 1 sparvu eng 18062 Jan 10 2007 PDQ_Report.c
-rwxr-xr-x 1 sparvu eng 7259 Feb 9 2007 PDQ_Lib_copy.h
-rw-r--r-- 1 sparvu eng 7581 Feb 28 2007 PDQ_Lib.h
-rw-r--r-- 1 sparvu eng 642 Feb 28 2007 Makefile.cygwin
-rw-r--r-- 1 sparvu eng 497 Feb 28 2007 Makefile.solaris
-rw-r--r-- 1 sparvu eng 525 Feb 28 2007 Makefile.gcc
-rwxr-xr-x 1 sparvu eng 573 Mar 14 2007 Makefile.macosx
Processing: ./Makefile.macosx
Processing: ./PDQ_Report.c
Processing: ./PDQ_Lib.h
Processing: ./PDQ_Lib_copy.h
Processing: ./PDQ_Exact.c
Processing: ./Makefile
Processing: ./Makefile.gcc
Processing: ./._Makefile.macosx
dos2unix: ./._Makefile.macosx not writable. Permission denied.
Processing: ./MVA_Approx.c
Processing: ./MVA_Canon.c
Processing: ./PDQ_Globals.c
Processing: ./Makefile.cygwin
Processing: ./PDQ_Utils.c
Processing: ./PDQ_Global.h
Processing: ./PDQ_Build.c
Processing: ./Makefile.solaris
Processing: ./MVA_Solve.c
Processing: ./debug.h
3. libpdq.a
Do we really need need the static library or not ?
Looks like {\tt setup.sh} for perl5 directory requires to have a {\tt libpdq.a} otherwise it won't compile clean.
But read on ...
sparvu@nereid>make
cc -ggdb -c MVA_Approx.c
cc: Warning: illegal option -db
"MVA_Approx.c", line 41: warning: implicit function declaration: debug
"MVA_Approx.c", line 44: warning: implicit function declaration: errmsg
"MVA_Approx.c", line 54: warning: implicit function declaration: resets
cc -ggdb -c MVA_Canon.c
cc: Warning: illegal option -db
"MVA_Canon.c", line 39: warning: implicit function declaration: debug
"MVA_Canon.c", line 55: warning: implicit function declaration: errmsg
"MVA_Canon.c", line 95: warning: implicit function declaration: typetostr
"MVA_Canon.c", line 108: warning: implicit function declaration: getjob_name
cc -ggdb -c MVA_Solve.c
cc: Warning: illegal option -db
"MVA_Solve.c", line 33: warning: implicit function declaration: debug
"MVA_Solve.c", line 42: warning: implicit function declaration: typetostr
"MVA_Solve.c", line 47: warning: implicit function declaration: errmsg
"MVA_Solve.c", line 53: warning: implicit function declaration: exact
"MVA_Solve.c", line 72: warning: implicit function declaration: approx
"MVA_Solve.c", line 88: warning: implicit function declaration: canonical
"MVA_Solve.c", line 124: warning: implicit function declaration: getjob_name
cc -ggdb -c PDQ_Build.c
cc: Warning: illegal option -db
"PDQ_Build.c", line 62: warning: implicit function declaration: debug
"PDQ_Build.c", line 65: warning: implicit function declaration: resets
"PDQ_Build.c", line 67: warning: implicit function declaration: errmsg
"PDQ_Build.c", line 187: warning: implicit function declaration: close
"PDQ_Build.c", line 208: warning: implicit function declaration: typetostr
"PDQ_Build.c", line 364: warning: implicit function declaration: getnode_index
"PDQ_Build.c", line 365: warning: implicit function declaration: getjob_index
cc -ggdb -c PDQ_Exact.c
cc: Warning: illegal option -db
"PDQ_Exact.c", line 55: warning: implicit function declaration: errmsg
cc -ggdb -c PDQ_Globals.c
cc: Warning: illegal option -db
cc -ggdb -c PDQ_Report.c
cc: Warning: illegal option -db
"PDQ_Report.c", line 76: warning: implicit function declaration: resets
"PDQ_Report.c", line 87: warning: implicit function declaration: errmsg
"PDQ_Report.c", line 187: warning: implicit function declaration: typetostr
"PDQ_Report.c", line 239: warning: implicit function declaration: debug
"PDQ_Report.c", line 309: warning: implicit function declaration: getjob_name
cc -ggdb -c PDQ_Utils.c
cc: Warning: illegal option -db
"PDQ_Utils.c", line 85: warning: implicit function declaration: getjob_index
ar -rv libpdq.a MVA_Approx.o MVA_Canon.o MVA_Solve.o PDQ_Build.o PDQ_Exact.o PDQ_Globals.o PDQ_Report.o PDQ_Utils.o
a - MVA_Approx.o
a - MVA_Canon.o
a - MVA_Solve.o
a - PDQ_Build.o
a - PDQ_Exact.o
a - PDQ_Globals.o
a - PDQ_Report.o
a - PDQ_Utils.o
ar: creating libpdq.a
ar: writing libpdq.a
# -ranlib -sc libpdq.a # MacOS X
ranlib libpdq.a # Cygwin
This creates a static library, using the cc compiler. However here I think we should use make -f Makefile.solaris
in order to get the shared library compiled.
4. perl5
$ ./setup.sh
make: Fatal error: Don't know how to make target `clean'
x - MVA_Approx.o
x - MVA_Canon.o
x - MVA_Solve.o
x - PDQ_Build.o
x - PDQ_Exact.o
x - PDQ_Globals.o
x - PDQ_Report.o
x - PDQ_Utils.o
Writing Makefile for pdq
cp pdq.pm blib/lib/pdq.pm
cc -c -D_LARGEFILE_SOURCE -D_FILE_OFFSET_BITS=64 -D_TS_ERRNO -xO3 -xspace -xildoff -DVERSION=\"\" -DXS_VERSION=\"\" -KPIC "-I/usr/perl5/5.8.4/lib/i86pc-solaris-64int/CORE" pdq_wrap.c
Running Mkbootstrap for pdq ()
chmod 644 pdq.bs
rm -f blib/arch/auto/pdq/pdq.so
LD_RUN_PATH="" cc -G MVA_Approx.o MVA_Canon.o MVA_Solve.o PDQ_Build.o PDQ_Exact.o PDQ_Globals.o PDQ_Report.o PDQ_Utils.o pdq_wrap.o -o blib/arch/auto/pdq/pdq.so
chmod 755 blib/arch/auto/pdq/pdq.so
cp pdq.bs blib/arch/auto/pdq/pdq.bs
chmod 644 blib/arch/auto/pdq/pdq.bs
Installing /export/home/sparvu/lib/site_perl/5.8.4/i86pc-solaris-64int/auto/pdq/pdq.so
Files found in blib/arch: installing files in blib/lib into architecture dependent library tree
Writing /export/home/sparvu///lib/site_perl/5.8.4/i86pc-solaris-64int/auto/pdq/.packlist
Appending installation info to /export/home/sparvu///lib/i86pc-solaris-64int/perllocal.pod
If we try to compile only the .so then STEP 4 fails, because it complains it can't find the static lib libpdq.a - Any ideas ?
5. Perl libraries.
Even as root ./setup.sh wont copy the libraries to the correct place.
# ./setup.sh
make: Fatal error: Don't know how to make target `clean'
x - MVA_Approx.o
x - MVA_Canon.o
x - MVA_Solve.o
x - PDQ_Build.o
x - PDQ_Exact.o
x - PDQ_Globals.o
x - PDQ_Report.o
x - PDQ_Utils.o
Writing Makefile for pdq
cp pdq.pm blib/lib/pdq.pm
cc -c -D_LARGEFILE_SOURCE -D_FILE_OFFSET_BITS=64 -D_TS_ERRNO -xO3 -xspace -xildoff -DVERSION=\"\" -DXS_VERSION=\"\" -KPIC "-I/usr/perl5/5.8.4/lib/i86pc-solaris-64int/CORE" pdq_wrap.c
Running Mkbootstrap for pdq ()
chmod 644 pdq.bs
rm -f blib/arch/auto/pdq/pdq.so
LD_RUN_PATH="" cc -G MVA_Approx.o MVA_Canon.o MVA_Solve.o PDQ_Build.o PDQ_Exact.o PDQ_Globals.o PDQ_Report.o PDQ_Utils.o pdq_wrap.o -o blib/arch/auto/pdq/pdq.so
chmod 755 blib/arch/auto/pdq/pdq.so
cp pdq.bs blib/arch/auto/pdq/pdq.bs
chmod 644 blib/arch/auto/pdq/pdq.bs
Installing /lib/site_perl/5.8.4/i86pc-solaris-64int/auto/pdq/pdq.so
Files found in blib/arch: installing files in blib/lib into architecture dependent library tree
Writing ////lib/site_perl/5.8.4/i86pc-solaris-64int/auto/pdq/.packlist
Appending installation info to ////lib/i86pc-solaris-64int/perllocal.pod
There shouldn't be many issues using PDQ under Linux, however I did receive the following email
From Rodrigo Campos (Brazil)
concerning SELinux
(not Swedish Linux, security enabled Linux).
Wednesday, December 19, 2007 at 21:32:59:
Just to let you know that when you try to run any Perl::PDQ script under a
SELinux enabled Linux distribution, you'll get the following error
(example):
[rcampos@localhost pdq_models]$ ./passport.pl
Can't load '/home/rcampos/lib/perl5/site_perl/5.8.8/i386-linux-thread-multi//auto/pdq/pdq.so'
for module pdq: /home/rcampos/lib/perl5/site_perl/5.8.8/i386-linux-thread-multi//auto/pdq/pdq.so:
cannot restore segment prot after reloc: Permission denied at
/usr/lib/perl5/5.8.8/i386-linux-thread-multi/DynaLoader.pm line 230.
at ./passport.pl line 4
Compilation failed in require at ./passport.pl line 4.
BEGIN failed--compilation aborted at ./passport.pl line 4.
In order to solve this problem you need to change the security context of
the pdq.so shared object by using the following command:
[rcampos@localhost ~]$ sudo chcon -t texrel_shlib_t /home/rcampos/lib/perl5/site_perl/5.8.8/i386-linux-thread-multi/auto/pdq/pdq.*
This information may be useful to others as many linux distros are coming
with SELinux enforced by default (in my case I use CentOS).
I don't know anything about ActivePerl (and I don't want to know), but I am very grateful
to these users for providing their installation instructions which may be helpful to
other ActivePerl users.
The following extensive notes were kindly provided by Tom Becker
who undertook a Herculean effort to get Perl::PDQ working under
ActiveState Perl on Windows XP. Reading this makes realize why I use
cygwin
on my Windows laptop (sonyXPress). But if you must use Active Perl,
this is the place to start. Thanks Tom!
This is my experience getting Perl::PDQ running in Windows XP. I have no C
skills, no UNIX skills, very limited Perl skills and am a PC user, not a
developer. All my information was gathered via Google and I hobbled together
all the required bits and pieces. All software used were free downloads.
Hopefully someone can drastically streamline my steps described below.
ENVIRONMENT: Windows XP Pro with SP1, Pentium 4 CPU, ActiveState Perl 5.8.3
(mswin32-x86-multi-thread) binary build 809 with 8 patches.
INSTALLATION:
1. Installed the PDQ_Perl package as recommended. My main
working folder was D:\pdq_perl\perl5\pdq\perl5. A lot of work was done in a
DOS window.
2. Downloaded NMAKE15 from Microsoft. This is version 1.5 of
nmake (the equivalent of the UNIX make command). Unzip this into the
..\perl5 folder referenced above. It will add nmake.exe and nmake.err. See
//download.microsoft.com/download/vc15/Patch/1.52/W95/EN-US/Nmake15.exe.
3. Run Perl makefile.pl.
4. ActiveState Perl calls for the cl compiler (Microsoft's C++
compiler). I downloaded a free copy of the Microsoft Visual C++ Toolkit
2003. Installed it and ran VCVARS32.BAT to set up the environmental
variables of LIB, PATH and INCLUDE.
5. Downloaded the Windows Server 2003 SP1 Platform SDK Full
Download and picked out windows.h and all associated include header files
and put them in the \Perl\lib\CORE folder. I did not install this product.
The full download is 400MB and has 16 separate 25MB files. A short cut is to
download two cab files from "Windows Server 2003 SP1 Platform SDK Full
Download": PSDK-FULL.6.cab and PSDK-FULL.7.cab. (Also get PSDK-FULL.15.cab
for a later step below.) The contents can be extracted with WINZIP. Extract
from the two files above: PSDK-SDK_Core_BLD-commom.0.cab,
PSDK-SDK_Core_BLD-commom.1.cab, PSDK-SDK_Core_BLD-commom.2.cab. Extract all
the contents of the above three files to one folder. Then sort by file name.
Copy and rename 27 header files to \perl\lib\core folder. Example:
windows_h.95CE7D7B_A68D_11D2_A852_00C04FC2A854 renamed to windows.h. Extract
the following:
BaseTsd.h
Guiddef.h
Imm.h
Mcx.h
PopPack.h
PshPack1.h
PshPack2.h
PshPack4.h
PshPack8.h
Reason.h
specstrings.h
StrAlign.h
Tvout.h
winbase.h
WinCon.h
WinDef.h
windows.h
WinError.h
WinGDI.h
WinNetWk.h
WinNls.h
WinNT.h
WinReg.h
WinSock.h
WinSvc.h
WinUser.h
WinVer.h
6. Modified d:\pdq_perl\perl5\pdq\lib\debug.h to remove the
three periods (...) in this statement:
#define g_debugf(fmt, args...) \
fprintf(stderr, "%s:%d " fmt, __FILE__, __LINE__,
##args)
7. Compiled all the c files in d:\pdq_perl\perl5\pdq\lib and
moved the .obj files to ..\perl5. There will be a warning message compiling
PDQ_Utils due to the modification to debug.h above.
8. Extract from PSDK-FULL.15.cab this file:
PSDK-SDK_MAC_BLD_X86-common.o.cab. From this file extract ODBC32.lib and
ODBCCP32.lib and move to ..\perl5.
9. Copy from the folder created in step 5 above (the extraction
of PSDK-SDK_Core_BLD-commom.0.cab, PSDK-SDK_Core_BLD-commom.1.cab,
PSDK-SDK_Core_BLD-commom.2.cab) and move the following to ..\perl5:
AdvAPI32.Lib
ComDlg32.Lib
Gdi32.Lib
Mpr.Lib
NetAPI32.Lib
Ole32.Lib
OleAut32.Lib
Shell32.Lib
User32.Lib
Uuid.Lib
Version.Lib
WinMM.Lib
WinSpool.Lib
WSock32.Lib
10. Downloaded and installed Microsoft .NET Framework SDK
Version 1.1 to get msvcrt.lib even though there was a copy of it in the
Windows Server 2003 SP1 Platform SDK ; it was for AMD64 and would not link.
After installation msvcrt.lib is found in \Program Files\Microsoft visual
Studio .NET\VC7\lib folder. Moved it to ...\perl5 with the other libs from
above.
11. Ran nmake.
12. Ran nmake install.
13. Ran the test.pl PDQ file to a successful completion.
RECOMMENDATIONS: (For the next pioneer):
1. Investigate why ActiveState Perl call for the cl.exe compiler (MS C++) in
the makefile. Perhaps a simpler compiler can be used.
2. Find out where all the Activestate Perl libs are that are displayed with
the perl -V command. It lists all the libs I had to search for, but were not
on my PC. So how does Activestate Perl use them?
3. Try to install the entire Windows Server 2003 SP1 Platform SDK FULL
DOWNLOAD. Maybe I overlooked msvcrt.lib for x86. However, I did inspect the
x86.msi file and it did reference the AMD64 file, which I tried but was
rejected by the link program. It will also simplify extracting all the
header files. But it is a 400MB download.
4. Get Microsoft to allow the redistribution of the headers and libs, rather
than forcing a download of the products I needed.
I (CS) use ActivePerl, but not cygwin. After some experiments I found a way of
installation of Perl::PDQ under win32 without extensive use of MS Tools.
Here is my approach:
1. Download nmake as descriped by Tom Becker
2. Download MinGW - Minimalist GNU for Windows (www.mingw.org) and install it
3. Open a DOS Shell and extend your path env variable so that mingw\bin is in front of perl.
set path=d:\programs\MinGW\mingw32\bin;d:\programs\MinGW\bin;D:\programs\perl\bin in my environment
4. change to the pdq\lib directory
5. use gcc (from mingw) to compile pdq c files to object files (for example MVA_Approx.o..)
6. copy or move the object files to the ..\pdq\perl5 directory
7. call perl MakeFile.PL
8. call nmake MakeFile
9. after a while this creates ..pdq\perl5\blib\arch\auto\pdq\pdq.dll
With this dll I can use Perl::PDQ without problems.
IMPORTANT:
If mingw\bin is not in the path, perl MakeFile.PL will create cl.exe calls (as Tom remarked).
But if Mingw\bin is in the path, the gcc compiler is used instead of MS.
So in general there is not a lot of difference between linux/cygwin and win32 for c-calls from perl.
BTW, there is a document (perlxstut.html) in the activeperl distribution,
which explains the general approach for Perl extensions.
Have built perl pdq.dll for Windows ... a real headache (even having
VisualStudio already installed). Just want to share the last problem I got,
not mentioned in the previous install experiences. After I built pdq.dll
and ran test.pl I got the following error:
C:\Projects\pdq42\perl5>perl test.pl
Can't load './pdq.dll' for module pdq: load_file:A dynamic link library (DLL) in
itialization routine failed at C:/Perl/lib/DynaLoader.pm line 230.
at test.pl line 6
Compilation failed in require at test.pl line 6.
BEGIN failed--compilation aborted at test.pl line 6.
The following command solved the problem: mt -manifest pdq.dll.manifest -outputresource:pdq.dll;2
The DLL is available for download. I did it
with ActivePerl 5.8.8 and VC80 ... not sure how portable it is (I see
dependencies from perl58.dll and msvcr80.dll).
If you would like to be notified by email about future PDQ updates, please fill out the
online form with your correct email address and select the heading
Notify for PDQ updates.
The same applies if you have changed your email address (e.g., changed employer).
A number of people have asked me for the C-language version of PDQ,
especially those using my 1998/2000 book
The Practical Performance Analyst.
The C-code of those PDQ models can be found in the directory examples/ppa_1998/.
The Perl scripts for the PDQ models discussed my 2005 book
Analyzing Computer System Performance with Perl::PDQ,
can now be found in the directory examples/ppdq_2005/.
A detailed synopsis of the PDQ functions in Perl is presented in Chapter 6.
