PDQ Software Distribution

This page was last updated on Mar 26, 2008

Current Distribution

Contents

1  Downloading

1. Java version of PDQ developed by Peter Harding
   
   
2. PHP version of PDQ developed by Italian student Samuel Zallocco
   
   
3. Threaded-server model and errata corrections developed by Neil Gunther
   
   
4. Better organization of the Perl and Python model directories
   
   

2  News

MSQ nodes:

I have implemented multiserver queues in the PDQ C-language only library, and it is now in beta test.

The goal is to allow you to create OPEN-circuit models (initially) by defining any sequence of M/M/m queues. Instead of introducing a completely new procedure, like PDQ_CreateMultiNode (or PDQ::CreateMultiNode described in Chapter 6 of the Perl::PDQ book), I have come up with a simpler idea. You will call PDQ_CreateNode() in the usual way but, by asserting a new MSQ flag to tell PDQ the node is multi-server queue, you then specify the integer number of servers (m) for that node. For example, an m = 10 multiserver PDQ node named "mserver" would be instantiated using the standard CreateNode procedure with the following arguments:

PDQ_CreateNode("mserver", 10, MSQ).

The PDQ generic report has also been altered to include the new MSQ node.

More details will be forthcoming as the implementation unfolds. Stay tuned.

Create functions becoming procedures:

Currently, the functions CreateOpen(), CreateClosed() each return an integer value corresponding to the total number of PDQ streams of work and CreateNode() returns an integer value corresponding to the total number of PDQ queueing nodes created. These returns are unnecessary and are also causing problems for porting.

In the next major release (PDQ 5.0 ?), the PDQ Create functions will become procedures i.e., they will no longer return a value. In ANSI parlance, they will become voids. To determine the count of PDQ streams or PDQ nodes, you will need to actively call two new functions:

• int streams = ... GetStreamsCount();
  
  
• int nodes = ... GetNodesCount();
  
  

Although most of you probably don't use these values, this is a heads-up to start planning for the change now.

3  Installing

3.1  UNIX Environments

1. Change to the /pdq subdirectory (cd pdq)
   
   
2. Change to the perl5 directory: cd perl5
   
   
3. Run the setup script: ./setup.sh
   
   
4. Go back up to the pdq directory: cd ..
   
   

• Check the PDQ version. Run the command ./Getversion to confirm that you have the same version number as above.
  
  
• Go into the /lib subdirectory and make sure you are using the appropriate Makefile or you may need to create your own---depending on your environment.
  
  

3.1.1  UNIX Installation Gotchas

• Line termination problems.
  Check the CR/LF conventions used in text files in your environment.
  
  
• Permissions problems.
  If you don't have root privileges, you will need to install the PERL module to a lib/ directory that is local to you.
  
  
• Pathing problems.
  The Perl interpreter has to be able to find the pdq.pm module. If it can't, the typical error message begins

  "Can't locate pdq.pm in ( contains: ...."

  To facilitate finding it on my MacOS X G4, for example, I had to export the UNIX environment variable PERL5LIB using the shell command:

  setenv PERL5LIB /Users/njg///lib/perl5/site_perl/5.8.6/darwin-.../

  You should then include this env var in your .cshrc or whatever login profile you use.
  Check this variable using the shell command: echo $PERL5LIB
  
  

3.2  Solaris Installations

• What compiler should be used: cc or gcc ?
  
  
• Do you really need to use libpdq.a? (See steps 3,4)
  
  
• Do you need to manually copy the Perl libraries into /usr/perl5?
  
  

1. Check pre-requisites
cc compiler:

$ which cc
/usr/bin/cc
$ cc -V 
cc: Sun C 5.9 SunOS_i386 Patch 124868-01 2007/07/12
usage: cc [ options] files.  Use 'cc -flags' for details

$ which make
/usr/bin/make

$ which dmake
/usr/bin/dmake


2. All delivered files are not proper under UNIX. Need to change the 
EOL character using dos2unix

Example, setup.sh:

#!/bin/sh^M
#^M
#  $Id: setup.sh,v 1.7 2006/04/08 23:56:15 zyx Exp $^M
#^M
^M
make clean^M
^M
ar xv ../lib/libpdq.a^M
^M
# swig -perl5 pdq.i^M
^M
perl Makefile.PL^M
^M
make install^M
^M

*** PERL5 Directory ***
$ for f in $(find . -type f); do print "Processing: $f"; dos2unix $f $f; done
Processing: ./.cvsignore
Processing: ./pdq_wrap_20070102.c
Processing: ./setup.sh
Processing: ./pdq.pm
Processing: ./pdq.i
Processing: ./pdq_wrap_old.c
Processing: ./pdq_wrap_x.c
Processing: ./Makefile.old
Processing: ./q
Processing: ./pdq_wrap.c
Processing: ./Makefile.PL
Processing: ./._test.out
dos2unix: ./._test.out not writable.  Permission denied.
Processing: ./README
Processing: ./test.pl
Processing: ./test.out

$ chmod 755 setup.sh


*** LIB Directory ***

$ pwd
/export/home/sparvu/workspace/PDQ/pdq42/lib
$ ls -lrt
total 195
-rwxr-xr-x   1 sparvu   eng          394 Feb 23  2004 debug.h
-rwxr-xr-x   1 sparvu   eng         3721 Oct  5  2004 MVA_Solve.c
-rwxr-xr-x   1 sparvu   eng         7515 Oct 30  2004 MVA_Approx.c
-rwxr-xr-x   1 sparvu   eng          634 Nov 19  2004 Makefile
-rwxr-xr-x   1 sparvu   eng         4515 May 10  2006 PDQ_Exact.c
-rwxr-xr-x   1 sparvu   eng         3366 May 13  2006 MVA_Canon.c
-rw-r--r--   1 sparvu   eng         1284 Jan  2  2007 PDQ_Global.h
-rw-r--r--   1 sparvu   eng         1302 Jan  2  2007 PDQ_Globals.c
-rw-r--r--   1 sparvu   eng        13166 Jan  2  2007 PDQ_Build.c
-rw-r--r--   1 sparvu   eng        15826 Jan 10  2007 PDQ_Utils.c
-rw-r--r--   1 sparvu   eng        18062 Jan 10  2007 PDQ_Report.c
-rwxr-xr-x   1 sparvu   eng         7259 Feb  9  2007 PDQ_Lib_copy.h
-rw-r--r--   1 sparvu   eng         7581 Feb 28  2007 PDQ_Lib.h
-rw-r--r--   1 sparvu   eng          642 Feb 28  2007 Makefile.cygwin
-rw-r--r--   1 sparvu   eng          497 Feb 28  2007 Makefile.solaris
-rw-r--r--   1 sparvu   eng          525 Feb 28  2007 Makefile.gcc
-rwxr-xr-x   1 sparvu   eng          573 Mar 14  2007 Makefile.macosx

Processing: ./Makefile.macosx
Processing: ./PDQ_Report.c
Processing: ./PDQ_Lib.h
Processing: ./PDQ_Lib_copy.h
Processing: ./PDQ_Exact.c
Processing: ./Makefile
Processing: ./Makefile.gcc
Processing: ./._Makefile.macosx
dos2unix: ./._Makefile.macosx not writable.  Permission denied.
Processing: ./MVA_Approx.c
Processing: ./MVA_Canon.c
Processing: ./PDQ_Globals.c
Processing: ./Makefile.cygwin
Processing: ./PDQ_Utils.c
Processing: ./PDQ_Global.h
Processing: ./PDQ_Build.c
Processing: ./Makefile.solaris
Processing: ./MVA_Solve.c
Processing: ./debug.h



3. libpdq.a 

Do we really need need the static library or not ?
Looks like {\tt setup.sh} for perl5 directory requires to have a {\tt libpdq.a} otherwise it won't compile clean. 
But read on ...

sparvu@nereid>make
cc -ggdb  -c  MVA_Approx.c
cc: Warning: illegal option -db
"MVA_Approx.c", line 41: warning: implicit function declaration: debug
"MVA_Approx.c", line 44: warning: implicit function declaration: errmsg
"MVA_Approx.c", line 54: warning: implicit function declaration: resets
cc -ggdb  -c  MVA_Canon.c
cc: Warning: illegal option -db
"MVA_Canon.c", line 39: warning: implicit function declaration: debug
"MVA_Canon.c", line 55: warning: implicit function declaration: errmsg
"MVA_Canon.c", line 95: warning: implicit function declaration: typetostr
"MVA_Canon.c", line 108: warning: implicit function declaration: getjob_name
cc -ggdb  -c  MVA_Solve.c
cc: Warning: illegal option -db
"MVA_Solve.c", line 33: warning: implicit function declaration: debug
"MVA_Solve.c", line 42: warning: implicit function declaration: typetostr
"MVA_Solve.c", line 47: warning: implicit function declaration: errmsg
"MVA_Solve.c", line 53: warning: implicit function declaration: exact
"MVA_Solve.c", line 72: warning: implicit function declaration: approx
"MVA_Solve.c", line 88: warning: implicit function declaration: canonical
"MVA_Solve.c", line 124: warning: implicit function declaration: getjob_name
cc -ggdb  -c  PDQ_Build.c
cc: Warning: illegal option -db
"PDQ_Build.c", line 62: warning: implicit function declaration: debug
"PDQ_Build.c", line 65: warning: implicit function declaration: resets
"PDQ_Build.c", line 67: warning: implicit function declaration: errmsg
"PDQ_Build.c", line 187: warning: implicit function declaration: close
"PDQ_Build.c", line 208: warning: implicit function declaration: typetostr
"PDQ_Build.c", line 364: warning: implicit function declaration: getnode_index
"PDQ_Build.c", line 365: warning: implicit function declaration: getjob_index
cc -ggdb  -c  PDQ_Exact.c
cc: Warning: illegal option -db
"PDQ_Exact.c", line 55: warning: implicit function declaration: errmsg
cc -ggdb  -c  PDQ_Globals.c
cc: Warning: illegal option -db
cc -ggdb  -c  PDQ_Report.c
cc: Warning: illegal option -db
"PDQ_Report.c", line 76: warning: implicit function declaration: resets
"PDQ_Report.c", line 87: warning: implicit function declaration: errmsg
"PDQ_Report.c", line 187: warning: implicit function declaration: typetostr
"PDQ_Report.c", line 239: warning: implicit function declaration: debug
"PDQ_Report.c", line 309: warning: implicit function declaration: getjob_name
cc -ggdb  -c  PDQ_Utils.c
cc: Warning: illegal option -db
"PDQ_Utils.c", line 85: warning: implicit function declaration: getjob_index
ar -rv libpdq.a MVA_Approx.o  MVA_Canon.o  MVA_Solve.o  PDQ_Build.o  PDQ_Exact.o  PDQ_Globals.o  PDQ_Report.o  PDQ_Utils.o
a - MVA_Approx.o
a - MVA_Canon.o
a - MVA_Solve.o
a - PDQ_Build.o
a - PDQ_Exact.o
a - PDQ_Globals.o
a - PDQ_Report.o
a - PDQ_Utils.o
ar: creating libpdq.a
ar: writing libpdq.a
# -ranlib -sc libpdq.a      # MacOS X
ranlib libpdq.a          # Cygwin


This creates a static library, using the cc compiler. However here I think we should use make -f Makefile.solaris
in order to get the shared library compiled. 


4. perl5 

$ ./setup.sh
make: Fatal error: Don't know how to make target `clean'
x - MVA_Approx.o
x - MVA_Canon.o
x - MVA_Solve.o
x - PDQ_Build.o
x - PDQ_Exact.o
x - PDQ_Globals.o
x - PDQ_Report.o
x - PDQ_Utils.o
Writing Makefile for pdq
cp pdq.pm blib/lib/pdq.pm
cc -c    -D_LARGEFILE_SOURCE -D_FILE_OFFSET_BITS=64 -D_TS_ERRNO -xO3 -xspace -xildoff    -DVERSION=\"\"  -DXS_VERSION=\"\" -KPIC "-I/usr/perl5/5.8.4/lib/i86pc-solaris-64int/CORE"   pdq_wrap.c
Running Mkbootstrap for pdq ()
chmod 644 pdq.bs
rm -f blib/arch/auto/pdq/pdq.so
LD_RUN_PATH="" cc  -G MVA_Approx.o MVA_Canon.o MVA_Solve.o PDQ_Build.o PDQ_Exact.o PDQ_Globals.o PDQ_Report.o PDQ_Utils.o pdq_wrap.o  -o blib/arch/auto/pdq/pdq.so      
chmod 755 blib/arch/auto/pdq/pdq.so
cp pdq.bs blib/arch/auto/pdq/pdq.bs
chmod 644 blib/arch/auto/pdq/pdq.bs
Installing /export/home/sparvu/lib/site_perl/5.8.4/i86pc-solaris-64int/auto/pdq/pdq.so
Files found in blib/arch: installing files in blib/lib into architecture dependent library tree
Writing /export/home/sparvu///lib/site_perl/5.8.4/i86pc-solaris-64int/auto/pdq/.packlist
Appending installation info to /export/home/sparvu///lib/i86pc-solaris-64int/perllocal.pod


If we try to compile only the .so then STEP 4 fails, because it complains it can't find the static lib libpdq.a - Any ideas ?


5. Perl libraries. 
Even as root ./setup.sh wont copy the libraries to the correct place. 

# ./setup.sh 
make: Fatal error: Don't know how to make target `clean'
x - MVA_Approx.o
x - MVA_Canon.o
x - MVA_Solve.o
x - PDQ_Build.o
x - PDQ_Exact.o
x - PDQ_Globals.o
x - PDQ_Report.o
x - PDQ_Utils.o
Writing Makefile for pdq
cp pdq.pm blib/lib/pdq.pm
cc -c    -D_LARGEFILE_SOURCE -D_FILE_OFFSET_BITS=64 -D_TS_ERRNO -xO3 -xspace -xildoff    -DVERSION=\"\"  -DXS_VERSION=\"\" -KPIC "-I/usr/perl5/5.8.4/lib/i86pc-solaris-64int/CORE"   pdq_wrap.c
Running Mkbootstrap for pdq ()
chmod 644 pdq.bs
rm -f blib/arch/auto/pdq/pdq.so
LD_RUN_PATH="" cc  -G MVA_Approx.o MVA_Canon.o MVA_Solve.o PDQ_Build.o PDQ_Exact.o PDQ_Globals.o PDQ_Report.o PDQ_Utils.o pdq_wrap.o  -o blib/arch/auto/pdq/pdq.so      
chmod 755 blib/arch/auto/pdq/pdq.so
cp pdq.bs blib/arch/auto/pdq/pdq.bs
chmod 644 blib/arch/auto/pdq/pdq.bs
Installing /lib/site_perl/5.8.4/i86pc-solaris-64int/auto/pdq/pdq.so
Files found in blib/arch: installing files in blib/lib into architecture dependent library tree
Writing ////lib/site_perl/5.8.4/i86pc-solaris-64int/auto/pdq/.packlist
Appending installation info to ////lib/i86pc-solaris-64int/perllocal.pod

3.3  Linux Installations

Wednesday, December 19, 2007 at 21:32:59:
Just to let you know that when you try to run any Perl::PDQ script under a
SELinux enabled Linux distribution, you'll get the following error
(example):

[rcampos@localhost pdq_models]$ ./passport.pl
Can't load '/home/rcampos/lib/perl5/site_perl/5.8.8/i386-linux-thread-multi//auto/pdq/pdq.so' 
for module pdq: /home/rcampos/lib/perl5/site_perl/5.8.8/i386-linux-thread-multi//auto/pdq/pdq.so: 
cannot restore segment prot after reloc: Permission denied at 
/usr/lib/perl5/5.8.8/i386-linux-thread-multi/DynaLoader.pm line 230.
 at ./passport.pl line 4
Compilation failed in require at ./passport.pl line 4.
BEGIN failed--compilation aborted at ./passport.pl line 4.


In order to solve this problem you need to change the security context of
the pdq.so shared object by using the following command:

[rcampos@localhost ~]$ sudo chcon -t texrel_shlib_t /home/rcampos/lib/perl5/site_perl/5.8.8/i386-linux-thread-multi/auto/pdq/pdq.*

This information may be useful to others as many linux distros are coming
with SELinux enforced by default (in my case I use CentOS).

3.4  WINDOWS with ActiveState Perl

1. Tom Becker - See Section 3.4.1
   
   
2. Christof Schmalenbach - See Section 3.4.2
   
   

3.4.1  Tom Becker's Notes

This is my experience getting Perl::PDQ running in Windows XP. I have no C
skills, no UNIX skills, very limited Perl skills and am a PC user, not a
developer. All my information was gathered via Google and I hobbled together
all the required bits and pieces. All software used were free downloads.
Hopefully someone can drastically streamline my steps described below. 

ENVIRONMENT: Windows XP Pro with SP1, Pentium 4 CPU, ActiveState Perl 5.8.3
(mswin32-x86-multi-thread) binary build 809 with 8 patches.

INSTALLATION:
	1.	Installed the PDQ_Perl package as recommended. My main
working folder was D:\pdq_perl\perl5\pdq\perl5. A lot of work was done in a
DOS window.

	2.	Downloaded NMAKE15 from Microsoft. This is version 1.5 of
nmake (the equivalent of the UNIX make command). Unzip this into the
..\perl5 folder referenced above. It will add nmake.exe and nmake.err. See
//download.microsoft.com/download/vc15/Patch/1.52/W95/EN-US/Nmake15.exe.

	3.	Run Perl makefile.pl.

	4.	ActiveState Perl calls for the cl compiler (Microsoft's C++
compiler). I downloaded a free copy of the Microsoft Visual C++ Toolkit
2003. Installed it and ran VCVARS32.BAT to set up the environmental
variables of LIB, PATH and INCLUDE.

         5.  Downloaded the Windows Server 2003 SP1 Platform SDK Full
Download and picked out windows.h and all associated include header files
and put them in the \Perl\lib\CORE folder. I did not install this product.
The full download is 400MB and has 16 separate 25MB files. A short cut is to
download two cab files from "Windows Server 2003 SP1 Platform SDK Full
Download": PSDK-FULL.6.cab and PSDK-FULL.7.cab. (Also get PSDK-FULL.15.cab
for a later step below.) The contents can be extracted with WINZIP. Extract
from the two files above: PSDK-SDK_Core_BLD-commom.0.cab,
PSDK-SDK_Core_BLD-commom.1.cab, PSDK-SDK_Core_BLD-commom.2.cab. Extract all
the contents of the above three files to one folder. Then sort by file name.
Copy and rename 27 header files to \perl\lib\core folder. Example:
windows_h.95CE7D7B_A68D_11D2_A852_00C04FC2A854 renamed to windows.h. Extract
the following:

BaseTsd.h
Guiddef.h
Imm.h
Mcx.h
PopPack.h
PshPack1.h
PshPack2.h
PshPack4.h
PshPack8.h
Reason.h
specstrings.h
StrAlign.h
Tvout.h
winbase.h
WinCon.h
WinDef.h
windows.h
WinError.h
WinGDI.h
WinNetWk.h
WinNls.h
WinNT.h
WinReg.h
WinSock.h
WinSvc.h
WinUser.h
WinVer.h

	6.	Modified  d:\pdq_perl\perl5\pdq\lib\debug.h to remove the
three periods (...) in this statement:
	 	#define g_debugf(fmt, args...) \
			fprintf(stderr, "%s:%d " fmt, __FILE__, __LINE__,
##args)

	7.	Compiled all the c files in d:\pdq_perl\perl5\pdq\lib and
moved the .obj files to ..\perl5. There will be a warning message compiling
PDQ_Utils due to the modification to debug.h above.

	8.	Extract from PSDK-FULL.15.cab this file:
PSDK-SDK_MAC_BLD_X86-common.o.cab. From this file extract ODBC32.lib and
ODBCCP32.lib and move to ..\perl5.

	9.	Copy from the folder created in step 5 above (the extraction
of PSDK-SDK_Core_BLD-commom.0.cab,   PSDK-SDK_Core_BLD-commom.1.cab,
PSDK-SDK_Core_BLD-commom.2.cab) and move the following to ..\perl5:

	 	AdvAPI32.Lib
		ComDlg32.Lib
		Gdi32.Lib
		Mpr.Lib
		NetAPI32.Lib
		Ole32.Lib
		OleAut32.Lib
		Shell32.Lib
		User32.Lib
		Uuid.Lib
		Version.Lib
		WinMM.Lib
		WinSpool.Lib
		WSock32.Lib
		
	10.	Downloaded and installed Microsoft .NET Framework SDK
Version 1.1 to get msvcrt.lib even though there was a copy of it in the
Windows Server 2003 SP1 Platform SDK ; it was for AMD64 and would not link.
After installation msvcrt.lib is found in \Program Files\Microsoft visual
Studio .NET\VC7\lib folder. Moved it to ...\perl5 with the other libs from
above. 

	11.	Ran nmake.

	12.	Ran nmake install.

	13.	Ran the test.pl PDQ file to a successful completion.

RECOMMENDATIONS: (For the next pioneer):
1. Investigate why ActiveState Perl call for the cl.exe compiler (MS C++) in
the makefile. Perhaps a simpler compiler can be used.

2. Find out where all the Activestate Perl libs are that are displayed with
the perl -V command. It lists all the libs I had to search for, but were not
on my PC. So how does Activestate Perl use them?

3. Try to install the entire Windows Server 2003 SP1 Platform SDK FULL
DOWNLOAD. Maybe I overlooked msvcrt.lib for x86. However, I did inspect the
x86.msi file and it did reference the AMD64 file, which I tried but was
rejected by the link program. It will also simplify extracting all the
header files. But it is a 400MB download.

4. Get Microsoft to allow the redistribution of the headers and libs, rather
than forcing a download of the products I needed. 

3.4.2  Christof Schmalenbach's Notes

1. Download nmake as descriped by Tom Becker
2. Download MinGW - Minimalist GNU for Windows (www.mingw.org) and install it
3. Open a DOS Shell  and extend your path env variable so that mingw\bin is in front of perl.
set path=d:\programs\MinGW\mingw32\bin;d:\programs\MinGW\bin;D:\programs\perl\bin in my environment
4. change to the pdq\lib directory
5. use gcc (from mingw) to compile pdq c files to object files (for example MVA_Approx.o..)
6. copy or move the object files to the ..\pdq\perl5 directory
7. call perl MakeFile.PL
8. call nmake MakeFile
9. after a while this creates ..pdq\perl5\blib\arch\auto\pdq\pdq.dll

With this dll I can use Perl::PDQ without problems.

IMPORTANT:
If mingw\bin is not in the path, perl MakeFile.PL will create cl.exe calls (as Tom remarked). 
But if Mingw\bin is in the path, the gcc compiler is used instead of MS.
So in general there is not a lot of difference between linux/cygwin and win32 for c-calls from perl.

BTW, there is a document (perlxstut.html) in the activeperl distribution, 
which explains the general approach for Perl extensions. 

3.4.3  Alex Podelko's Notes

C:\Projects\pdq42\perl5>perl test.pl
Can't load './pdq.dll' for module pdq: load_file:A dynamic link library (DLL) in
itialization routine failed at C:/Perl/lib/DynaLoader.pm line 230.
 at test.pl line 6
Compilation failed in require at test.pl line 6.
BEGIN failed--compilation aborted at test.pl line 6.

• ms235560.aspx
  
  
• ms235591.aspx
  
  

4  Get Notified About Updates

5  Where's Waldo? (Anyone C-een him?)

6  User Guide

7  Feedback