Software projects are getting larger and more complicated year after year stressing the need for efficient and reliable tools and agile project management methods. Widely used and recognized tools in Open Source development, CVS and Bugzilla, have been proven to be suitable for most kind of software projects and have lately been accompanied in many projects with an emerging document and content management system, Wiki. These tools are usually used separately, rarely integrated with each other to enable easier understanding and management of the whole process flow from specifications and code to releases and maintenance. Integrated tools would also provide more effective and seamless environment for software development with improved quality and productivity.
This document provides a detailed configuration guide to integrate incrementally several Open Source development tools creating a seamless development enviroment without unnecessary complexity or overhead. Bugzilla/CVS/MediaWiki are all linked to each other and they together with mailing lists and various CVS utilities provide easily accessible way to see changes to documentation and source code, handling of bug reports, and people contributing to projects without any additional tools or preparations in matter of minutes whenever needed. All this is achieved by using three special magic identifiers to create references to other tools. The identifier BugID: creates references to Bugzilla, VCS: creates references to version control system, CVS, and Wiki: creates references to Wiki. Whenever writing a Wiki document, a bug report, or a CVS commit message, these identifiers can be used and they are then automatically turned into hyperlinks pointing to the tools referenced. This enables smooth browsing from specifications to code, from code to bug reports, and back from bug reports to code changes and documentation. Additionally, all activities in projects are archived in mailing lists, CVS utilities provide graphical statistics, and bug reports contain references to actual changes or documentation that permitted closing the report. The instructions below proceed incrementally by configuring one tool after another until the whole environment is set up. Provided screenshots illustrate results of integration. These instructions can be used of course separately to set up only parts of the presented environment or to integrate with the tools already in use. Reader is expected to have basic knowledge about the tools and their purpose.
The following instructions to configure Mailman mailing list
manager are based on Red Hat
packaged Mailman RPMs that
have some paths different compared to official Mailman releases but
otherwise the instructions can be used with any version of Mailman. It
is of course necessary to run
sendmail (or some other MTA) at the server where Mailman archives are located.
First, Apache is
configured to redirect queries to
/mailman to a list info
page. This setting is recommended for all Mailman installations and
requires only adding one directive to the file
/etc/httpd/conf.d/mailman.conf that is installed from the
Mailman RPM. Obviously, you
must change the hostname to match your local environment.
RedirectMatch ^/mailman[/]*$ http://www.dom.ain/mailman/listinfo
This setting will be effective after the service
/sbin/service httpd restart
Actual Mailman configuration begins by setting Mailman site password with the command:
Main configuration file for Mailman is
/etc/mailman/mm_cfg.py. There are dozens of configuration
directives that could be set but here only two mandatory settings and
few recommended ones are listed. The mandatory settings are:
DEFAULT_URL_HOST = 'www.dom.ain' DEFAULT_EMAIL_HOST = 'dom.ain'
The settings must, of course, be adjusted to match your local
environment. If you use
DEFAULT_EMAIL_HOST then you should also add the following
line just after the previous directives:
Other recommended and/or useful settings include the following:
ALLOW_SITE_ADMIN_COOKIES = Yes # For secure intranets only DEFAULT_CHARSET = 'utf-8' DEFAULT_MAX_MESSAGE_SIZE = 4096 # kB # These are needed if archives are accessed only by using HTTPS DEFAULT_URL_PATTERN = 'https://%s/mailman/' PUBLIC_ARCHIVE_URL = 'https://%(hostname)s/pipermail/%(listname)s'
To update Mailman configuration after configuring needed directives the following command must be executed:
Next, for administration purposes the mailing list
must be created:
The command will ask for an e-mail address for person running the list.
The e-mail address must contain proper host and domain parts also for
local users so enter an e-mail address like
[email protected]. Follow the provided
instructions on how to update the file
properly process e-mails sent to the list. After updating
/etc/aliases, the alias database must be updated with the
mailman that updates the archives when needed
can now be started:
/sbin/chkconfig mailman on /sbin/service mailman start
Now the actual lists for Bugzilla and CVS can finally be created. Two lists will be used, one
for archiving Bugzilla activities and one for CVS activities. For Bugzilla activities, the
list shall be
prj-bugs and for CVS commits
To create a new mailing list open the URL
http://www.dom.ain/mailman/create in a
browser and follow the instructions to create the list. Mandatory
information that must be provided when creating a list is the name of
the list, initial list owner e-mail address, and initial list password.
For other settings, default values are usually suitable. Note that after
creating the mailing lists the file
updated as instructed in the e-mails sent to the list creator and after
the update the command newaliases(1) must be run.
After both the lists have been created configuration should be changed
to allow anyone to post to the lists. Especially with the list
prj-vcs this eases maintenance, as there is no need to
duplicate changes to CVS
rights in the mailing list configuration. To allow all postings to a
list, visit list's admin page and go to
Filter and change settings By default, should new list
member postings be moderated? to No and Action
to take for postings from non-members for which no explicit action is
defined. to Accept.
The last step is to enable the magic identifiers in the mailing list
archives. Applying this
patch and restarting the
mailman service is all what is
needed to enable them. The patch was generated against Mailman 2.1.5
but it should apply also against later versions in the 2.1.x series.
Once done, the mailing list configuration is ready.
Both Bugzilla and MediaWiki use MySQL database to store data so before
installing Bugzilla or MediaWiki, MySQL must be installed and
configured. These instructions provide only quick steps to get MySQL 4
or later ready for Bugzilla and MediaWiki, for any performance
optimizations or other advanced adjustments you should refer to MySQL
documentation. MySQL should be installed from a package provided
with your distribution. Note that before configuring MySQL the MySQL
database server (
mysqld) must be running.
The very first configuration step after MySQL installation is to set
root user password. This can be done with the
following command. Note that the command will fail if the password is
already set so this command can be executed more than once if in doubt.
mysqladmin -uroot password <root-password>
Actually, this is all what is needed for MediaWiki as its installer will
set up the rest. But for Bugzilla the user
bugs must also
mysql -uroot -p<root-password> mysql GRANT SELECT, INSERT, UPDATE, DELETE, INDEX, ALTER, CREATE, LOCK TABLES, CREATE TEMPORARY TABLES, DROP, REFERENCES ON bugs.* TO [email protected] IDENTIFIED BY '<bugs-password>'; FLUSH PRIVILEGES;
Those are the only mandatory configurations needed for Bugzilla and
MediaWiki but you probably also want to configure
start during system bootup. Other MySQL configuration options can be
studied from the MySQL documentation. For reference, however, few of the
most useful options are also provided here for download. To apply these settings add them to
/etc/mysql.cnf into the
configuration usually requires a bit of work but the actual procedure is
rather simple. Before installing Bugzilla, make sure that you have
installed Perl and MySQL is configured. If Perl is not installed, use
your distribution's packages to install it. Then, download latest stable Bugzilla
release, unpack the downloaded archive, and move the extracted
/var/www/html/bugzilla. To start Bugzilla
configuration, enter the Bugzilla directory, fix owner of the files and
directories, and run the
checksetup.pl script with the
cd /var/www/html/bugzilla chown -R root:root . * ./checksetup.pl --check-modules
The script will most likely complain about missing Perl modules. You
must install all the required Perl modules. Whether you should
also install all or some of the optional modules mentioned depends on
many things. If you eat Perl for breakfast and already know how to
install Perl modules then it is surely no problem to install all the
optional modules, too. But if you just want to get Bugzilla up and
running and perhaps later investigate the features provided by the
optional modules then just follow the instructions provided by the
checksetup.pl script to install the required modules. Note
that some of the modules may be found from your distribution's package
collections. If the module installation gets complicated, you may find
some help from the Bugzilla documentation. When
checksetup.pl is finally happy, rerun the script without
checksetup.pl will instruct you to edit the file
localconfig. Most likely the only setting you want to
$db_pass. Set it to match the
user password you used earlier during MySQL
configuration. When done, rerun the script
checksetup.pl again without any parameters:
The script should proceed until it asks for an administrator. Please
enter a valid e-mail address, administrator's real name, and a proper
checksetup.pl finishes, one final step is
remaining; Apache must be configured to allow CGI scripts to be executed in the Bugzilla
installation directory. This can be done by adding the following lines
to Apache configuration file, usually
<Directory /var/www/html/bugzilla> AddHandler cgi-script .cgi Options +Indexes +ExecCGI DirectoryIndex index.cgi AllowOverride Limit </Directory>
Setting will be effective after the service
httpd has been
/sbin/service httpd restart
Bugzilla is now installed and mostly configured and can be used with a
browser at the URL
http://www.dom.ain/bugzilla/. Log in as administrator and
Parameters page. There are several configuration
options that could be tweaked. At least the following two options should
maintainer should contain valid e-mail address
of the Bugzilla installation maintainer and
contain the URL of the
Bugzilla installation. There are also two seducing CVS related options:
functionality enabled by those options is of very little or no use at
all; they only provide more contexts to patches attached to bug reports
if the code is in the CVS repository. Certainly you may experiment with those
options but it is recommended to ignore them. More information about all
the options can be found from the Bugzilla documentation.
After saving parameter changes, you must add two new users for
integration purposes. As administrator go to administration page
Users->add a new user and then create users
[email protected] and
[email protected]. You may want to use
descriptive real names like
Commits but more importantly disable the accounts by stating
their purpose in the
Disable field. The accounts will be
used to send bug activity e-mails to the
list and to append CVS
commits log messages to bug reports.
To enable automatic bug activity e-mails
Default CC List
must be set for each component for which the feature is wanted.
Components can be found under the menu
Product. Same account can be reused for as many components as
wanted and typically all components of a product have the same CC list when used for activity e-mail
generation. Example product
TestProduct and its component
TestComponent that are automatically created during
Bugzilla installation can be used to see how the automated e-mailing
works by setting
Default CC List account of the
[email protected]. Whenever a bug is created
TestComponent or status of a
TestComponent bug changes an e-mail is sent to the
prj-bugs mailing list and archived.
The last required step is to enable the magic identifiers in Bugzilla. Applying this patch is all what is needed to enable them. The patch was generated against Bugzilla 3.0.2 but it should apply also against later Bugzilla 3.x releases.
At this stage you might want to test your Bugzilla installation a bit by
creating a test bug or two against the example product
TestProduct and see how linking with the magic identifiers
and automated e-mails work. You may also want to adjust some other
options, such as the e-mail templates and time tracking group, in the
Parameters page to better suit your needs. After Bugzilla
seems to be your liking, you can proceed to the next
Available screenshots: commit message in Bugzilla.
Basic set up of a CVS repository is rather
straightforward task. The most important configuration setting with
CVS is the environment
CVSROOT that defines the used CVS repository. The
configuration presented here uses the directory
/var/lib/cvsroot as the repository and expects that all
CVS users belong to the
users. More information about other settings can be
found from the CVS manual.
To initialize a new repository, create necessary files, and set file permissions correctly the following commands must be executed:
export CVSROOT=/var/lib/cvsroot mkdir -p /var/lib cvs init chgrp users $CVSROOT chmod 2775 $CVSROOT touch $CVSROOT/CVSROOT/history chgrp users $CVSROOT/CVSROOT/history chmod 0664 $CVSROOT/CVSROOT/history touch $CVSROOT/CVSROOT/val-tags chgrp users $CVSROOT/CVSROOT/val-tags chmod 0664 $CVSROOT/CVSROOT/val-tags mkdir -p /var/lock/cvs chgrp users /var/lock/cvs chmod 2775 /var/lock/cvs cvs co CVSROOT sed -ie 's,#LockDir=.*$,LockDir=/var/lock/cvs,g' CVSROOT/config cvs ci -m "Use a lock dir." CVSROOT/config echo y | cvs release -d CVSROOT
These commands set
CVSROOT, create and initialize the
repository and a lock directory, create files for CVS operations, and make sure
the files have sane permissions. More information about these files can
be found from CVS
documentation but during this integration deeper knowledge about
these files is not essential.
To add a test module
prj with some dummy content the
following commands should be executed:
mkdir -p prj/test cd prj echo foo > foo echo bar > test/bar cvs import -m "Imported sources." prj prj-orig start cd .. rm -rf prj
Now, to start the actual integration work, CVSweb is
configured and patched so that the CVS repository can be accessed with a browser and the
magic identifiers are shown as links. Either install CVSweb using the
official sources or use a CVSweb package available for your
distribution. Then, apply two minor patches to the script
cvsweb.cgi. These patches were generated against version
3.0.6 of the cvsweb.cgi script but they should apply also to other
CVSweb versions. The first patch applied is to prevent error messages with new
files added to the repository and the second is to enable using
Wiki: magic identifiers. In addition, some minor changes to the
cvsweb.conf configuration file must be done so that CVSweb
properly uses the repository created earlier and the magic identifiers
are enabled. The configuration file patch was generated against the original
3.0.6 configuration file but it should apply also to other versions.
There also some other changes that will be explained
a bit later. For convenience, both the patched files are also
available: cvsweb.cgi and cvsweb.conf.
Before proceeding any further, please make sure that at this stage your CVS repository is properly set up, checkouts and checkins can be done, and that the repository can be accessed with a browser via CVSweb. You should also do some test commits using the magic identifiers in the log message to see that CVSweb produces correct links for them. An example log entry could be something like:
During integration testing a problem was reported at BugID: 1 that although the specification at Wiki: The_Spec says that we should do bar, VCS: prj/foo was doing foo. Fixed to do bar.
If both the CVS
repository and the CVSweb interface are working properly then it is time
to begin perhaps the most difficult part of the integration. The CVS repository will be
configured to send commit e-mails to the
list and to integrate with Bugzilla. All this is done by modifying the
administrative files under the directory
and using some integration scripts. Since there are many
interdependencies between the files it is probably best to do all
required changes at a go, then test the changes, and fix any problems
that might arise.
To begin, checkout the directory
CVSROOT and add or replace
the following files: checkoutlist, commit_prep, commitinfo, log_accum_bugzillafied, loginfo, rcsinfo, rcstemplate. It is recommended to use
also these two optional files: cvswrappers, users. Note that the following files
must be adjusted to match your local environment
before checking them in:
loginfo, and, optionally,
users. The purpose of each file is briefly described in
comments in the beginning of a file. Below are some additional notes
that should help to understand the way the integration is done. Note
that before checking in the files please read this section entirely and
also make sure that the scripts
log_accum_bugzillafied are executable:
chmod a+rx commit_prep log_accum_bugzillafied
checkoutlist simply lists additional files that should be
treated as CVS
administrative files in addition to the standard administrative files.
rcstemplate are used to add a help
text that is shown when writing a commit message.
cvswrappers simply lists file types that should be handled
as binary files.
users is used when automatically
generating changelogs as explained in the next
section. It does not harm to add it even without real content.
The following files, originally used in projects at sourceware.org, are in the heart of the integration and may require a bit more attention to understand them completely. However, deeper knowledge is not necessarily required since all the files contain a short configuration section that needs to be adjusted only to match your local settings; the actual code should require no changes at all.
commitinfo defines a command to be executed before commits.
commit_prep is defined and it just records the last
directory with commits for use by
loginfo defines a command to be executed after the commits.
log_accum_bugzillafied is defined and it sends
commit e-mails to specified addresses. Note that the script
log_accum_bugzillafied itself does not yet add data to
Bugzilla database, it will be done by the script processing the commit
e-mail sent to Bugzilla. Of these scripts,
loginfo need most
adjusting as they contain e-mail addresses, URLs, and other settings that are always
project specific. Nevertheless, the required changes should be rather
obvious after some inspection.
Since the script
log_accum_bugzillafied sends the commit
messages as e-mails, they must be received and processed properly. The
processing is done by the script
email_in.pl that is part
of the Bugzilla installation. The script must be slightly patched to enable it
to find all required Perl modules. The following line must be added to
/etc/aliases to automatically invoke the script
sendmail receives commit e-mails. Note that the
recipient must match to the one defined in the script
And, once again, the command newaliases(1) must be run after
the alias database has been updated. After the update,
sendmail executes the script for all e-mails sent to the
bugzilla. However, by default
is not allowed to run random commands while processing e-mails so the
script must be listed as an allowed command. Also the permissions in the
Bugzilla installation must be changed so that the user
sendmail is running is allowed to read and
execute all the needed scripts. The following commands allow
sendmail to invoke the script
change permissions so that the user
cd /etc/smrsh ln -s /var/www/html/bugzilla/email_in.pl . cd /var/www/html/bugzilla chmod g+w data/mimedump-tmp chown -R apache:mail . *
It must be noted that running again the script
checksetup.pl or editing Bugzilla parameters will change
the file permissions so that they must be adjusted again. However, as
these are rather rare events it does not cause too much maintenance
burden but this is well worth keeping in mind in case of any integration
After all these changes, you can finally commit the modifications in
CVSROOT directory. Before actually testing the integration,
please make sure that all Perl modules required by
email_in.pl have been installed. This can be verified by
running the script from the command line. If any error messages are
printed, some modules, such as MIME-tools, are probably missing. If no
errors are printed, the Bugzilla/CVS integration is complete! You may now experiment with
commits containing the magic identifiers and see how the commit messages
are appended to bug reports alongside with hyperlinks to actual changes
in CVS. If everything
seems to be working you are ready to proceed to the
next section. But be assured, the rest will be a pleasure cruise
compared to the struggle you have been through so far.
Available screenshots: commit message in repository web interface.
This section provides quick instructions on how to use some rather
useful utilities to generate statistics from your CVS repository. Although they
are not absolutely required from the integration point of view, they all
fall into the
low-hanging fruit category and are thus worth
CvsGraph is a
utility to make graphical representations of CVS repository. It is integrated with CVSweb and
it very likely can be found from your distribution's package collection.
Compiling it from source should also be trivial, after compilation just
cvsgraph binary in a directory contained in
PATH. Configuration is simple and can be done by copying this file as
/etc/cvsgraph.conf. See CvsGraph pages for
more information about CvsGraph and example graphics.
CVSHistory is a utility
written in Python
that is used to track development activity on CVS using software projects. It is integrated
with CVSweb and its installation is simply done by copying the script
cvshistory.cgi from the CVSHistory package to your web
server's CGI directory,
/var/www/cgi-bin. Then set the
CONFIGFILE location in the script as
/etc/cvshistory.conf and copy this configuration file as
/etc/cvshistory.conf. That is all needed for installation,
a History link is now present in the CVSweb interface that
allows easy activity tracking. See CVSHistory homepage for more
information about CVSHistory.
Perl scripts used to
generate formatted changelogs based on CVS activities. Installation is simply done by placing
the scripts in a directory contained in
/usr/local/bin. There are several command line parameters
that could be used but for convenience a
shell script is provided that creates GNU/HTML/XML formatted changelogs from the CVS repository and also a nice
web page to access them. Please refer to the documentation of the
scripts for more information about their usage. Note that the shell
script requires the file
$CVSROOT/CVSROOT/users to be
created and if it contains valid mappings
for names those are used when generating the changelogs. As can be seen
from the script the statictics web page created by default is
a tool written in Java that is used
to create exhaustive amount of statistics from projects in a CVS repository. Installation
is simply done by placing the JAR file
from the StatCVS package to a proper directory, e.g.,
/usr/local/share and running it with the help of
java(1) command. There are several command line parameters
that could be used but for convenience the previously provided shell script also contains needed
commands to create StatCVS statistics that also contain links to the
CVSweb interface. Please refer to StatCVS documentation
for more information about its usage and example statistics.
Other related CVS utilities that might be mentioned here are LXR, a general purpose source code indexer and cross-referencer, and Bonsai, a tool that lets one to perform queries on the contents of a CVS archive.
Available screenshots: links to statistics generated from the repository.
MediaWiki is the
easiest piece in this puzzle so installing it will take only a while.
Before installation, make sure that you have MySQL
configured and PHP
and PHP's MySQL module
installed. They are usually included in distributions' package
collections and require no or very little configuration. Once PHP is enabled, get latest stable MediaWiki
release, unpack the downloaded archive, and move the extracted
/var/www/html/wiki. You must then prepare
directory permissions for configuration with the commands:
cd /var/www/html/wiki chown -R root:root . * chmod 0777 config
MediaWiki configuration is then done with a browser simply by opening
http://www.dom.ain/wiki/ and answering to configuration
queries. Sufficient help for most queries is available at the
configuration page so most of the items are not covered here.
Shared memory caching would provide performance speedup but in this basic configuration, no caching is available. For all e-mail related queries you should answer depending on your own needs. Since MediaWiki automatically maintains history of recent changes and also provides them as an RSS feed so e-mailing them to a mailing list would not be very useful. Finally, in the Database config section the provided default values are suitable. For the field DB password use a proper password that is different from the MySQL root password that was set during MySQL installation. The previously set MySQL root password must then be entered into the field Superuser password. Note that you must also set the option Use superuser accounts. For Database charset it is recommended to select Backward-compatible UTF-8.
After hitting the Install! wait a minute until the next page is loaded. Check that it says Installation successful! and to finish the configuration enter the following commands:
cd /var/www/html/wiki mv config/LocalSettings.php . chmod 0700 config
MediaWiki is now installed and configured. To allow usage of the magic identifiers also in Wiki you need only to apply this patch. The patch was generated againt MediaWiki version 1.11.0 but it should apply also against later versions in the 1.11.x series. After applying the patch to the installed MediaWiki the magic identifiers turn into hyperlinks.
Now, after all your hard work, the integration should be finally completed, congratulations! Bugzilla/CVS/MediaWiki are all linked to each other and they together with the mailing lists and various CVS utilities provide easily accessible way to see how projects have evolved and improved. Changes to documentation and source code, handling of change requests and people involved in the project can be found out without any additional tools or preparations in matter of minutes whenever needed.
Available screenshots: magic identifiers in a Wiki page.
This document provided a detailed guide to integrate Bugzilla, CVS, MediaWiki, several CVS utilities, and mailing lists together. After integration, these standard tools provide an effective and seamless software configuration management (SCM) environment for professional software development allowing developers and managers to keep on track of projects and quality issues. Environment where everything from specifications and code to bug reports and code modifications are all integrated and easily accessible improves tremendously both quality and productivity of a project. Environment could be further enhanced with other tools such as a source tree monitoring system, a test case management system, a code review system, an integrated IDE, and other similar features. In a corporate environment, access restrictions and more sophisticated user management methods might also be needed. These kinds of configurations are, however, outside of scope of this document.
All the configuration files mentioned are browsable and available for download.
Comments and feedback can be sent to oss(at)segetech.com. We would kindly like to note that we cannot promise a personal reply to all e-mails received, especially to those asking for help in some unrelated configuration problems. Inquiries about possible cooperation should be done via addresses listed at our Contact page.
This document was last updated on 2007-11-13.