Server requirements
- Apache 1.3 or higher
- PHP 4.x or higher (said to have worked on PHP5, but not tested so beware)
- MySQL 4.0 or higher
Python and Sentry Requirements
External Python Module Requirements
these packages should be installed in Python's site-packages directory.
Environment Requirements
The script file itself (sentry.py) can be placed in any directory for which there is execute access. Sentry needs a location for its configuration file (sentry.conf) and persistent data. Sentry can write reports as html, so write access to a directory accessible from a web server would helpful, though not required.
The distribution includes a sample configuration file called sentry-dist.conf. To use it, copy it to the target configuration directory and name the file sentry.conf. If this configuration file lives somewhere other than the same directory as the application, the application must be told where it is:
./sentry.py -c /somepath/sentry.conf
Database Requirements
This program depends on having both read and write access to the Bouncer MySQL database instance.
Check out the code
If you are checking out a particular snapshot, check it out from tags/ or branches/ instead of trunk.
Set up the database
mysql -u user -D db -h host.foo.com -p < ./sql/bouncer.sql
Apache configuration
- Python must have a handler in Apache for loader.py – usually run as a CGI script.
- Typically ./php is the webroot for the vhost, and ./python is accessed through an alias.
- Since Python files are .conf, you should disallow access to *.conf for Bouncer's vhost.
PHP setup
Setting up config.php
In the ./php/cfg/ directory, you'll find config-dist.php. Copy this to config.php.
cp php/cfg/config-dist.php php/cfg/config.php
The config.php document has comments in it that explain what each item is for. Let's take a look:
define('FILEPATH','/var/www/download'); define('WEBPATH',''); define('LIB',FILEPATH.'/lib'); define('INC',FILEPATH.'/inc'); define('HEADER',INC.'/header.php'); define('FOOTER',INC.'/footer.php'); define('DBHOST', ''); define('DBNAME', ''); define('DBUSER', ''); define('DBPASS', ''); define('LOGGING',1);
- FILEPATH - this is the root of the php side of things
- WEBPATH - this is the relative path for the application. If your application was foo.com/bar/ then your WEBPATH would be "/bar".
- DB* - database config stuff, the usual
- LOGGING - this determines whether or not you want to update download counts when people get bounced with the bounce script or the download form. If you turn this off, the only way you'll be able to track hits is by munging apache logs. By default this is turned on, because it's a relatively cheap hit on the DB.
Displaying errors when you first start out
PHP doesn't need any specific configuration on the server end of things. Keep in mind that by default PHP typically has display_errors and error_reporting dumbed down. You might want to turn these on when setting up the admin interface for the first time to catch configuration problems:
ini_set('display_errors',1);
ini_set('error_reporting',E_ALL);
Sentry setup
Options
Sentry 2 has many options. These options can be set using environment variables, a configuration file or via the command line. Most options also have default values. Environment variables override the defaults. The configuration file overrides environment variables and the defaults. Command line options override all three.
The command line options are specified by either the short form, a single hyphen followed by a character, or the long form, a double hyphen followed by the command name. Some options require a parameter. For the short form, parameters can be specified with or with out a space after the option. For the long form, the option and the parameter are separated by either a space or the equal sign.
The configuration file is simple a text file listing the long form option names, one to a line. The double hyphen is not used in the configuration file. Parameters are separated from the options with either a space or equal sign.
Environment variables use the long form option names.
-b, --benchmarking
report on the time that each test took
-C, --checklevel
specify the thoroughness of the check:
1 - mirrortest: check only that mirrors respond
2 - filetest: check that files exist on mirrors
3 - fulltest: check of all files with hash code comparison
4 - newmirrorsandfilestest: full test on new mirrors and files only
5 - full test on one randomly selected file on each mirror (default: 1)
-z, --compactVisualization
leave out empty columns from the visualization on a per page basis
-c, --config
specify the location and name of the config file (default: ./sentry.conf)
-d, --databaseName
the name of the database within the server (default: )
-p, --databasePassword
the database user's password (default: )
-h, --databaseServer
the host name of the database server (default: )
-u, --databaseUserName
the login name for the database (default: )
-E, --email
send email reports of actions
-L, --emailCriticalLogEvents
send email for any critical errors that cause the program to quit
-q, --emailPassword
the password for the email user (default: )
-r, --emailReturnAddress
the return address for email messages sent (default: )
-S, --emailSMTPServer
the SMTP server to use for email messages (default: )
-s, --emailSendToAddress
the address to which email messages are sent (default: )
-e, --emailUserName
the name of the email user (default: )
-a, --hashReadChunkSize
the number in bytes to read at a time from a file during hash calculation (default: 131072)
-?, --help
print this message
-G, --hoursInVisualization
number of hours of history to show in the visualization (default: 8)
-H, --hoursOfHistory
number of hours to hold in the history for the visualization (default: 8)
-v, --htmlReportMirrorsPerPage
number of mirrors per page on the HTML visualization (default: 5)
-l, --loggingLevel
write what's going on to stdout - the lower the number, the more verbose it is.
1-19 - debug
20-39 - general info
40- - critical errors (default: 40)
-4, --notifyFileActionBadNoChange
notify when a file is bad and does not change state
-2, --notifyFileActionBadToGood
notify when a file goes from bad to good
-3, --notifyFileActionGoodNoChange
notify when a file is good and does not change state
-1, --notifyFileActionGoodToBad
notify when a file goes from good to bad
-8, --notifyMirrorActionBadNoChange
notify when a mirror is bad and does not change state
-6, --notifyMirrorActionBadToGood
notify when a mirror goes from bad to good
-7, --notifyMirrorActionGoodNoChange
notify when a mirror is good and does not change state
-5, --notifyMirrorActionGoodToBad
notify when a mirror goes from good to bad
-n, --numberOfThreads
specify the number of threads to use (default: 5)
-P, --persistentDataPath
the filesystem location to store persistent data (default: ./)
-t, --timeout
the time in seconds to wait for servers to respond (default: 5)
-V, --visual
produce a visualization using html
-w, --webpath
base path for writing html files (default: ./)
Sample Configuration File
#-------------------
# s e n t r y 2 . 0
#-------------------
#
# test configuration file
email
emailUserName=lohnk
emailSMTPServer=mySMTPserver
emailPassword=mypassword
emailReturnAddress=emailFromSentry@mydomain.org
emailSendToAddress=iNeedToKnowWhatsHappening@mydomain.org
emailCriticalLogEvents
databaseName=theDatabaseName
databaseUserName=theDatabaseUser
databasePassword=thePassword
databaseServer=theDatabaseServerHostName
notifyFileActionGoodToBad
notifyFileActionBadToGood
#notifyFileActionGoodNoChange
notifyFileActionBadNoChange
notifyMirrorActionGoodToBad
notifyMirrorActionBadToGood
#notifyMirrorActionGoodNoChange
#notifyMirrorActionBadNoChange
visual
benchmarking
numberOfThreads=30
persistentDataPath=/glacier/home/lohnk/project/bouncer/000/bouncer/
webpath=/glacier/home/lohnk/project/bouncer/000/bouncer/html/
Scheduling Recurring Executions
Sentry 2 is meant to be run around the clock. The various tests should be distributed throughout the day. The mirror test is the lightest weight test and could be run most frequently. The full test (3) is generally the heaviest test.
Before deploying Sentry, it would be wise to carefully plan the scheduling. Sentry will not allow multiple instances of the Full Test, the New Mirrors and Files Test, and Random Full Test (3, 4, 5) to run simultaneously. Since each of these tests downloads complete files, disallowing simultaneous executions limits the load on the mirrors. The Mirror Test and File Test (1, 2) can be run during any other test type. Since these tests only touch mirrors, they do not add significantly to a mirror's load.
Depending on the the number of files and mirrors in the database, the Full Test can take significant time. Using the sample Mozilla database with fifty-nine mirrors, the full test took six hours to run from the developer's workstation. Sentry is a multithreaded program. Each thread is allocated a mirror and is responsible for downloading all the file s for that mirror. The six hour test time used fifty-nine threads. This means that the slowest mirror took all six hours to deliver its files. The fastest mirror took only a few minutes. It is important that Sentry continue to run the Mirror Test during the interim period between the end of the fastest mirror and the end of the slowest mirror. Because that interim period may be several hours, we need to insure that Bouncer reacts if a mirror were to go down.
*/15 * * * * .../sentry.py -C 1
5 * * * * .../sentry.py -C 2
8 0 * * * .../sentry.py -C 3
8 6 * * * .../sentry.py -C 4
Every fifteen minutes, the [MirrorTest] is run. At five minutes after every hour, it runs the [FileTest]. At eight minutes after midnight, it runs a Full Test. At eight minutes after 6AM, it runs the New Mirrors/Files test.
Reporting Options - What did Sentry do?
Sentry is designed with many options for reporting its actions. It can send reports to stdout, stderr, email, and Web pages, as well as log its actions in a database.
Types of Events
Sentry logs two types of actions: Mirror Events and File Events. The terms "actions" and "events" are used interchangeably. These two event types record the changes in status of individual mirrors and files. Sentry will generate a file event each time it tests a file. Mirror events are generated after all scheduled file tests for that mirror have completed. Each of these events also contains auxiliary information about the details of the event including any error messages.
- [FileActionBadNoChange]
- [FileActionBadToGood]
- [FileActionGoodNoChange]
- [FileActionGoodToBad]
- [MirrorActionBadNoChange]
- [MirrorActionBadToGood]
- [MirrorActionGoodNoChange]
- [MirrorActionGoodToBad]
By default, Sentry does not report anything. While it is probably important to be notified of file or mirrors being marked "bad" after having been "good", it is of lesser importance to know about files or mirrors being "good" and remaining that way after testing. Each of the events has a corresponding command line switch (or environment or config file entry). Adding "--notifyFileActionGoodToBad" to the command line will set Sentry to notify on occurrence of that event. Each of the events has a corresponding switch.
stdout and email
By default, these events are reported to stdout. They may be redirected to email instead by setting the "email" option via a command line switch, environment variable or within the configuration file.
stderr
The reporting that can occur on stderr are useful for debugging. These are controlled via the "loggingLevel" command line switch. A value is given for "loggingLevel" in the range of 1 to 40. The lower the number the more detailed the logging of what Sentry is doing. The "--notify*" options for Mirror and File events have no effect on this type of reporting.
database
The database reporting is controlled by the "--hoursOfHistory" option. If the "hoursOfHistory" is given a non-zero value, Sentry will log every Mirror and File event without regard to the "--notify*" switches. Sentry will hold this information for the number of hours specified.
web visualization
The Web reporting generates HTML based visualizations of the Mirror and File events based on the logged events in the database. These pages will be generated if the "--visual" switch is present. The pages will be generated at the location specified by "--webpath" It doesn't necessarily have to use the entire log in the database, the number hours of log information con be restricted by the "--hoursInVisualization" switch. Since it can generate massive Web pages, the pages can be divided by the "--htmlReportMirrorsPerPage" switch. Finally, because it is possible for more than one test to run simultaneously and to preserve the even flow time across the report, Sentry will interleave tests in the report based on the time stamp of each event. This means that a test may span several columns. Depending on how the report is paginated, there may be entirely empty columns on a given page. The "--compactVisualization" switch will remove blank columns to compact the display.
Loader setup
Unable to render {include} Couldn't find a page to include called: Loader Setup
