Add a set of scripts that can be used to build and upload the

distributable binaries for all platforms all from a single
command-line on a single machine.  Will probably also be used for a
daily build cron job.


git-svn-id: https://svn.wxwidgets.org/svn/wx/wxWidgets/trunk@27791 c3d73ce0-8a6f-49c7-b76d-6d57e0e08775
This commit is contained in:
Robin Dunn
2004-06-14 19:53:50 +00:00
parent 96acd0c161
commit 36e910973a
10 changed files with 630 additions and 4 deletions

View File

@@ -1,6 +1,7 @@
.DS_Store
.emacs.desktop
.gdb_history
BUILD
MANIFEST
_build_dmg
_build_rpm

View File

@@ -2,7 +2,7 @@
# ----------------------------------------------------------------------
FLAGS="USE_SWIG=1 SWIG=e:/projects/SWIG-cvs/swig.exe"
FLAGS="USE_SWIG=1 SWIG=$PROJECTS/SWIG-cvs/swig.exe"
# Use non-default python?
case $1 in

View File

@@ -46,6 +46,7 @@ wxPython/demo/bmp_source
wxPython/demo/data
wxPython/demo/dllwidget
wxPython/distrib
wxPython/distrib/all
wxPython/distrib/mac
wxPython/distrib/mac/wxPythonOSX
wxPython/distrib/mac/wxPythonOSX/resources

View File

@@ -0,0 +1,34 @@
The collection of scripts in this directory are an attempt to fully
automate the build of the wxPython source and binary packages on all
build platforms. It does this through creative use of shared folders
on network drives, and ssh commands to the remote machines. So this
will likly only work in my somewhat unique environment.
The goal here is to be able to start a build on one machine and have
it take care of all the steps, including moving the source tarball to
the other build machines, initiating the build there, and collecting
the results. Depending on the type of build, (see below) the results
may be copied to a public server for others to play with.
Types of builds:
dry-run
Nothing extra is done with the build, this is just for
my own testing.
daily
The version number is temporarily adjusted to include a
datestamp, and if the build is successful the results
are copied to a daily build folder on starship.
release-cantidate
The results are uploaded to the previews foler on
starship if the build is successful.
The master script in this folder is "make-all" which will setup and
control the whole process. The other scripts are what are run on each
build machine, most of which will also call out to other scripts that
already exist, etc.

370
wxPython/distrib/all/build-all Executable file
View File

@@ -0,0 +1,370 @@
#!/bin/bash
# ---------------------------------------------------------------------------
# Master build script for building all the installers and such on all the
# build machines in my lab, and then distributing the results as needed.
# ---------------------------------------------------------------------------
set -o errexit
#set -o xtrace
# ---------------------------------------------------------------------------
# Some control variables...
# the local spot that we put everything when done, before possibly copying
# to remote hosts
STAGING_DIR=./BUILD
# host name of the machine to use for windows builds
WIN_HOST=cyclops
# local dir (from the poerspecitve of the master machine) of the shared
# build dir on the windows machine, plus how to mount it there. If the
# [u]mount commands are empty then WIN_SHARED will be used directly.
WIN_SHARED=./tmp/mnt
WIN_MOUNT="smbmount //$WIN_HOST/BUILD $WIN_SHARED -o credentials=/etc/samba/auth.cyclops.robind,dmask=755,fmask=644"
WIN_UMOUNT="smbumount $WIN_SHARED"
# Where is that dir from the remote machine's perspective?
WIN_SHARED_REMOTE=/c/BUILD
# Same as the above
OSX_HOST=bigmac
OSX_SHARED=./tmp/mnt/BUILD
OSX_MOUNT="smbmount //$OSX_HOST/robind ./tmp/mnt -o credentials=/etc/samba/auth.bigmac.robind,dmask=755,fmask=644"
OSX_UMOUNT="smbumount ./tmp/mnt"
OSX_SHARED_REMOTE=/Users/robind/BUILD
# Alsmost the same... See below
LINUX_HOST=rh9
LINUX_SHARED=/stuff/temp/BUILD
LINUX_MOUNT=
LINUX_UMOUNT=
LINUX_SHARED_REMOTE=/stuff/temp/BUILD
# Upload server locations
UPLOAD_HOST=starship.python.net
UPLOAD_DAILY_ROOT=/home/crew/robind/public_html/wxPython/daily
UPLOAD_PREVIEW_ROOT=/home/crew/robind/public_html/wxPython/preview
# ---------------------------------------------------------------------------
# functions
function usage {
echo ""
echo "Usage: $0 [command flags...]"
echo ""
echo "build types:"
echo " dryrun Do the build, but don't copy anywhere (default)"
echo " daily Do a daily build, copy to starship"
echo " release Do a normal release build, copy to starship"
echo ""
echo "optional command flags:"
echo " 2.2 Build for Python 2.2 (default=off)"
echo " 2.3 Build for Python 2.3 (default=on)"
echo " all Build for all supported Python versions"
echo ""
echo " skipsource Don't build the source archives, use the ones"
echo " already in the staging dir."
echo " onlysource Exit after building the source archives"
echo " skipwin Don't do the remote Windows build"
echo " skiposx Don't do the remote OSX build"
echo " skiplinux Don't do the remote Linux build"
echo " skipclean Don't do the cleanup step on the remote builds"
echo ""
}
# ---------------------------------------------------------------------------
# Make sure we are running in the right directory. TODO: make this
# test more robust. Currenly we just test for the presence of
# 'wxPython' and 'wx' subdirs.
if [ ! -d wxPython -o ! -d wx ]; then
echo "Please run this script from the root wxPython directory."
exit 1
fi
# Set defaults and check the command line options
KIND=dryrun
PYVER=2.3
skipsource=no
onlysource=no
skipwin=no
skiposx=no
skiplinux=no
skipclean=no
for flag in $*; do
case $flag in
dryrun) KIND=dryrun ;;
daily) KIND=daily ;;
release) KIND=release ;;
2.2) PYVER=2.2 ;;
2.3) PYVER=2.3 ;;
all) PYVER="2.2 2.3" ;;
skipsource) skipsource=yes ;;
onlysource) onlysource=yes ;;
skipwin) skipwin=yes ;;
skiposx) skiposx=yes ;;
skiplinux) skiplinux=yes ;;
skipclean) skipclean=yes ;;
help) usage; exit 1 ;;
*) echo "Unknown flag \"$flag\""
usage
exit 1
esac
done
# ensure the staging area exists
if [ ! -d $STAGING_DIR ]; then
mkdir -p $STAGING_DIR
fi
# Figure out the wxPython version number, possibly adjusted for being a daily build
if [ $KIND = daily ]; then
DAILY=`date +%Y%m%d` # should it include the hour too? 2-digit year?
echo $DAILY > DAILY_BUILD
fi
VERSION=`python -c "import setup;print setup.VERSION"`
#echo VERSION=$VERSION
#exit 0
# ---------------------------------------------------------------------------
# Make the sources and other basic stuff.
if [ $skipsource != yes -o $onlysource = yes ]; then
# clean out the local dist dir
rm -f dist/*
# Regenerate the reST docs
echo "Regenerating the reST docs..."
cd docs
for x in *.txt; do
docutils-html $x `basename $x .txt`.html
done
cd -
# build the doc and demo tarballs
distrib/makedemo
distrib/makedocs
# make the source tarball
distrib/makerpm 2.3 skipclean skiprpm gtk2
# make the source RPMs
for ver in $PYVER; do
distrib/makerpm $ver skipclean skipcopy skiptar srpm
distrib/makerpm $ver skipclean skipcopy skiptar srpm gtk2
done
# Copy everything to the staging dir
echo "Moving stuff to $STAGING_DIR..."
rm -f dist/*.spec
mv dist/* $STAGING_DIR
for doc in CHANGES BUILD INSTALL MigrationGuide default; do
cp docs/$doc.* $STAGING_DIR
done
# cleanup
echo "Cleaning up..."
rm -f dist/*
rm -rf _build_rpm
fi
if [ $KIND = daily ]; then
rm DAILY_BUILD
fi
if [ $onlysource = yes ]; then
exit 0
fi
# ---------------------------------------------------------------------------
# Windows build
if [ $skipwin != yes ]; then
echo "-=-=- Starting Windows build..."
# mount the shared folder?
if [ -n "$WIN_MOUNT" ]; then
echo "Mounting shared folder..."
$WIN_MOUNT
fi
# Copy the src file
echo "Copying source file..."
cp $STAGING_DIR/wxPythonSrc-$VERSION.tar.gz $WIN_SHARED
# Untar it on the remote machine, and then run the build script
echo "Unarchiving source file on $WIN_HOST..."
ssh $WIN_HOST "cd $WIN_SHARED_REMOTE && tar xzf wxPythonSrc-$VERSION.tar.gz && rm wxPythonSrc-$VERSION.tar.gz"
echo "Running build script on $WIN_HOST..."
wxdir=$WIN_SHARED_REMOTE/wxPythonSrc-$VERSION
cmd=$wxdir/wxPython/distrib/all/build-windows
ssh $WIN_HOST "cd $wxdir && $cmd $wxdir $WIN_SHARED_REMOTE $skipclean $VERSION $PYVER"
echo "Fetching the results..."
cp $WIN_SHARED/wxPythonWIN32* $STAGING_DIR
ssh $WIN_HOST "cd $WIN_SHARED_REMOTE && rm wxPythonWIN32*"
# unmount?
if [ -n "$WIN_UMOUNT" ]; then
echo "Unmounting shared folder..."
$WIN_UMOUNT
fi
fi
# ---------------------------------------------------------------------------
# OSX build
if [ $skiposx != yes ]; then
echo "-=-=- Starting OSX build..."
# mount the shared folder?
if [ -n "$OSX_MOUNT" ]; then
echo "Mounting shared folder..."
$OSX_MOUNT
fi
# Copy the src file
echo "Copying source files..."
cp $STAGING_DIR/wxPythonSrc-$VERSION.tar.gz $OSX_SHARED
cp $STAGING_DIR/wxPythonDocs-$VERSION.tar.gz $OSX_SHARED
cp $STAGING_DIR/wxPythonDemo-$VERSION.tar.gz $OSX_SHARED
# Untar it on the remote machine, and then run the build script
echo "Unarchiving source file on $OSX_HOST..."
ssh $OSX_HOST "cd $OSX_SHARED_REMOTE && tar xzf wxPythonSrc-$VERSION.tar.gz && rm wxPythonSrc-$VERSION.tar.gz"
echo "Running build script on $OSX_HOST..."
wxdir=$OSX_SHARED_REMOTE/wxPythonSrc-$VERSION
cmd=$wxdir/wxPython/distrib/all/build-osx
ssh $OSX_HOST "cd $wxdir && $cmd $wxdir $OSX_SHARED_REMOTE $skipclean $VERSION $PYVER"
echo "Fetching the results..."
cp $OSX_SHARED/wxPythonOSX* $STAGING_DIR
ssh $OSX_HOST "cd $OSX_SHARED_REMOTE && rm wxPythonOSX*"
# unmount?
if [ -n "$OSX_UMOUNT" ]; then
echo "Unmounting shared folder..."
$OSX_UMOUNT
fi
fi
# ---------------------------------------------------------------------------
# Linux build
# This build is optional, check if the target machine is up and
# running
if [ $skiplinux != yes ]; then
if ping -q -c1 -w1 $LINUX_HOST > /dev/null; then
# the ping succeeded
skiplinux=no
else
# the ping failed, skip the build
skiplinux=yes
echo "-----------------------------------------------------------------"
echo "The $LINUX_HOST machine is offline, skipping the binary RPM build."
echo "-----------------------------------------------------------------"
fi
fi
if [ $skiplinux != yes ]; then
echo "-=-=- Starting Linux build..."
# The remote linux build is a bit different than the others. The
# SRPMs will have already been built in the initial source
# building steps, the only thing that the remote build needs to
# do is an "rpmbuild --rebuild" for each package. So we'll just
# copy the build script over and execute it, there is no need to
# unpack the tarball and most of the other steps...
echo "Copying source files..."
mkdir -p $LINUX_SHARED
cp $STAGING_DIR/wxPython*.src.rpm $LINUX_SHARED
scp distrib/all/build-linux root@$LINUX_HOST:/tmp > /dev/null
ssh root@$LINUX_HOST "cd /tmp && ./build-linux /tmp/wx $LINUX_SHARED_REMOTE $skipclean $VERSION $PYVER"
echo "Fetching the results..."
cp $LINUX_SHARED/wxPythonGTK*.i[0-9]86.rpm $STAGING_DIR
rm -r $LINUX_SHARED
fi
# ---------------------------------------------------------------------------
# Final disposition of build results...
if [ $KIND = dryrun ]; then
# we're done
exit 0
fi
if [ $KIND = daily ]; then
destdir=$UPLOAD_PREVIEW_ROOT/$DAILY
echo "Copying to the starship at $destdir..."
ssh $UPLOAD_HOST "mkdir -p $destdir"
scp $STAGING_DIR/* $UPLOAD_HOST:/$destdir
echo "Cleaning up staging dir..."
rm $STAGING_DIR/*
rmdir $STAGING_DIR
# TODO: something to remove old builds from starship, keeping
# only N days worth
# TODO: Send email to wxPython-dev?
exit 0
fi
if [ $KIND = release ]; then
echo "Copying to the local file server..."
destdir=/stuff/Development/wxPython/dist/$VERSION
mkdir -p $destdir
cp $STAGING_DIR/* $destdir
echo "Copying to the starship..."
destdir=$UPLOAD_PREVIEW_ROOT/$VERSION
ssh $UPLOAD_HOST "mkdir -p $destdir"
scp $STAGING_DIR/* $UPLOAD_HOST:/$destdir
echo "Cleaning up staging dir..."
rm $STAGING_DIR/*
rmdir $STAGING_DIR
exit 0
fi
# ---------------------------------------------------------------------------

View File

@@ -0,0 +1,64 @@
#!/bin/bash
# ---------------------------------------------------------------------------
# Build the wxPython source RPMs on a Linux box. This is normally called
# from build-all but it should be able to be used standalone too...
#
# The command line must have the following parameters:
#
# 1. the path to the base of the wx source tree
# 2. the path of where to put the resulting RPMs
# 3. skipclean flag (yes|no)
# 4. the VERSION
# 5. the remaining args are the versions of Python to build for
#
# ---------------------------------------------------------------------------
set -o errexit
#set -o xtrace
echo "-=-=-=- Hello from $HOSTNAME -=-=-=-"
if [ $# -lt 5 ]; then
echo "Usage: $0 WXDIR DESTDIR SKIPCLEAN VERSION PYVER..."
exit 1
fi
WXDIR=$1
DESTDIR=$2
SKIPCLEAN=$3
VERSION=$4
shift;shift;shift;shift
PYVER=$@
# Since this is probably a VMWare guest, make sure that the date and
# time are correct
ntpdate gate.alldunn.com
# In this case $WXDIR is where we will build at, but the source tree
# won't be there like the other builds. The source RPMs will be in
# $DESTDIR, and we'll put the binary RPMs back there when done.
echo "Preparing $WXDIR..."
mkdir -p $WXDIR
cd $WXDIR
rm -rf *
for ver in $PYVER; do
echo "Building the RPMs for Python $ver..."
myrpmbuild --rebuild $DESTDIR/wxPythonGTK-py$ver-$VERSION-1.src.rpm
myrpmbuild --rebuild $DESTDIR/wxPythonGTK2-py$ver-$VERSION-1.src.rpm
done
echo "Copying RPMs to $DESTDIR..."
cp wxPythonGTK*.i[0-9]86.rpm $DESTDIR
cd $DESTDIR
if [ $SKIPCLEAN != yes ]; then
echo "Cleaning up..."
rm -rf $WXDIR
fi
echo "-=-=-=- Goodbye! -=-=-=-"

50
wxPython/distrib/all/build-osx Executable file
View File

@@ -0,0 +1,50 @@
#!/bin/bash
# ---------------------------------------------------------------------------
# Build wxWidgets and wxPython on a OSX (Panther) box. This is normally
# called from build-all but it should be able to be used standalone too...
#
# The command line must have the following parameters:
#
# 1. the path to the base of the wx source tree
# 2. the path of where to put the resulting installers
# 3. skipclean flag (yes|no)
# 4. the VERSION
# 5. the remaining args are the versions of Python to build for
#
# ---------------------------------------------------------------------------
set -o errexit
#set -o xtrace
echo "-=-=-=- Hello from $HOSTNAME -=-=-=-"
if [ $# -lt 5 ]; then
echo "Usage: $0 WXDIR DESTDIR SKIPCLEAN VERSION PYVER..."
exit 1
fi
WXDIR=$1
DESTDIR=$2
SKIPCLEAN=$3
VERSION=$4
shift;shift;shift;shift
PYVER=$@
echo "Invoking wxPythonOSX build script..."
cd $WXDIR/wxPython
export TARBALLDIR=$DESTDIR
mkdir -p dist
distrib/mac/wxPythonOSX/build panther inplace skipclean
echo "Copying installers to $DESTDIR..."
cp dist/*.dmg $DESTDIR
cd $DESTDIR
if [ $SKIPCLEAN != yes ]; then
echo "Cleaning up..."
rm -r $WXDIR
fi
echo "-=-=-=- Goodbye! -=-=-=-"

View File

@@ -0,0 +1,100 @@
#!/bin/bash
# ---------------------------------------------------------------------------
# Build wxWidgets and wxPython on a Windows box. This is normally called
# from build-all but it should be able to be used standalone too...
#
# The command line must have the following parameters:
#
# 1. the path to the base of the wx source tree
# 2. the path of where to put the resulting installers
# 3. skipclean flag (yes|no)
# 4. the VERSION
# 5. the remaining args are the versions of Python to build for
#
# ---------------------------------------------------------------------------
set -o errexit
#set -o xtrace
echo "-=-=-=- Hello from $HOSTNAME -=-=-=-"
if [ $# -lt 5 ]; then
echo "Usage: $0 WXDIR DESTDIR SKIPCLEAN VERSION PYVER..."
exit 1
fi
WXDIR=$1
DESTDIR=$2
SKIPCLEAN=$3
VERSION=$4
shift;shift;shift;shift
PYVER=$@
# WXDIR is the cygwin path, WXWIN is the DOS path
WXWIN_OLD=$WXWIN
WXWIN=`cygpath -w $WXDIR`
export WXWIN
# Fix the PATH. (Why is this needed??)
PATH=/usr/local/bin:/usr/bin:/bin:/usr/X11R6/bin:/home/robind/bin:.:$WXDIR/lib/vc_dll:$PATH
export PATH
# change to the right spot and copy our build scripts
cd $WXDIR/build/msw
cp $WXDIR/wxPython/distrib/msw/.m* .
# replace some settings in setup0.h and write to setup.h
cat > .my.sedexpr <<EOF
# s/wxDIALOG_UNIT_COMPATIBILITY *1/wxDIALOG_UNIT_COMPATIBILITY 0/g
s/wxUSE_DEBUG_CONTEXT *0/wxUSE_DEBUG_CONTEXT 1/g
s/wxUSE_MEMORY_TRACING *0/wxUSE_MEMORY_TRACING 1/g
s/wxUSE_DIALUP_MANAGER *1/wxUSE_DIALUP_MANAGER 0/g
s/wxUSE_GLCANVAS *0/wxUSE_GLCANVAS 1/g
s/wxUSE_POSTSCRIPT *0/wxUSE_POSTSCRIPT 1/g
s/wxUSE_AFM_FOR_POSTSCRIPT *1/wxUSE_AFM_FOR_POSTSCRIPT 0/g
s/wxUSE_DISPLAY *0/wxUSE_DISPLAY 1/g
EOF
cat $WXDIR/include/wx/msw/setup0.h | sed -f .my.sedexpr > $WXDIR/include/wx/msw/setup.h
rm .my.sedexpr
echo "Building the wx DLLs..."
.make hybrid
.make hybrid-uni
echo "Building the wx tools..."
.make_tools
# cheat and just copy the .CHM files from the regular project dir
mkdir -p $WXDIR/docs/htmlhelp
cp `cygpath $WXWIN_OLD/docs/htmlhelp`/*.chm $WXDIR/docs/htmlhelp
echo "Building wxPython and installers..."
cd $WXDIR/wxPython
for ver in $PYVER; do
echo $ver
b $ver d USE_SWIG=0
b $ver h USE_SWIG=0
b $ver r USE_SWIG=0
b $ver d UNICODE=1 USE_SWIG=0
b $ver h UNICODE=1 USE_SWIG=0
b $ver r UNICODE=1 USE_SWIG=0
done
echo "Building the developer package..."
4nt /c distrib/makedev.bat $VERSION
echo "Copying installers to $DESTDIR..."
mv dist/wxPythonWIN32* $DESTDIR
cd $DESTDIR
if [ $SKIPCLEAN != yes ]; then
echo "Cleaning up..."
rm -r $WXDIR
fi
echo "-=-=-=- Goodbye! -=-=-=-"

View File

@@ -150,6 +150,12 @@ if [ -z "${skipcopy}" ]; then
cp -Rpf --link ${wxdir}/$d ${tarver} #> /dev/null 2>&1
fi
done
# # and tex2rtf too
# mkdir ${tarver}/utils
# cp -Rpf --link ${wxdir}/utils/tex2rtf ${tarver}/utils
# # tex2rtf needs these files
# mkdir ${tarver}/samples
# cp -Rpf --link ${wxdir}/samples/sample.* ${tarver}/samples
# now do the same thing for wxPython, skipping it's build dirs and such
for dir in `grep -v '#' ${wxdir}/wxPython/distrib/DIRLIST`; do

View File

@@ -11,8 +11,8 @@ nmake -f makefile.vc BUILD=release $@
cp vc_msw/wxrc.exe $WXWIN/lib/vc_dll/wxrc.exe
cd $WXWIN/utils/tex2rtf/src
nmake -f makefile.vc BUILD=release $@
cp vc_msw/tex2rtf.exe $WXWIN/lib/vc_dll/tex2rtf.exe
# cd $WXWIN/utils/tex2rtf/src
# nmake -f makefile.vc BUILD=release $@
# cp vc_msw/tex2rtf.exe $WXWIN/lib/vc_dll/tex2rtf.exe