Multithreading Model

Sentry 2 uses threads in Python. Some would say that Python's thread model is hobbled by the Global Interpreter Lock (GIL) so it will not properly take advantage of a multiprocessor system. Since this program is IO bound rather than CPU bound, that limitation should not be a problem.

Sentry's threads can be put into three categories: the master thread, the task threads and the action thread.

The master thread of Sentry 2 is in charge of assigning mirrors to the task threads for testing. Using the database, it fetches all the mirrors eligible for testing. Cycling through that list, it submits tasks to the task queue. A task is an assignment to download and test the files on a single mirror. A task includes: the mirror information, a list of files expected to be on that mirror and a reference to a function that can perform the desired test.

On the other end of the task queue, we've got a mob of worker threads waiting to grab tasks. These are called the task threads. They are in charge of doing the actual downloads and integrity tests. As each thread gets its task, it executes the test function, using the mirror and file information as parameters. The test function cycles through each of the files applying its test. The function reports the results of its tests by submitting an action function and the mirror/file information onto the action queue. There are action functions for each of the possible results of a test. Once an action function is queued, the task thread either moves on to the next file for that mirror, or, if all the files are done, it goes back to the task queue for a new mirror to test.

The action thread services the action queue. It takes the actions submitted by the worker threads and does them. These actions potentially updating the database as well as collecting data for reporting purposes.

When the main thread has queued all the mirrors, it posts special tasks on the task queue, one for each thread. These special tasks can be thought of as death tokens. When a thread picks up the death token from the task queue, it quits. Once all the task threads have died, the main thread posts a death token on the action queue. When the action thread dies, the program ends.

Object Model

Sentry uses a Python flavor of 'mixins' that change the inheritance hierarchy at run time to create objects. These objects then use the Cooperative Classesconcept. In cooperative classes, rather than a subclass function overriding a super class function, the subclass function invokes the superclasses function of the same name. This way, execution ascends the inheritance hierarchy.

The initial inheritance hierarchy looks like this:

After initialization, Sentry will rearrange the hierarchy to make sure it has all the features it needs to do it's job. For example, if the user selected that they want benchmarking, emailing and complete checksum testing for all files, the inheritance hierarchy would be arranged like this:

Once the rearrangement is made, an instance of this class is instantiated and execution jumps to it.

The class [EmailingMirrorIntegrityChecker] added a function called notifyOfActionsTaken. This function overrides the function of the same name in super class [MirrorIntegrityChecker]. In the super class this dumps a list of actions that the program took to standard out. The [EmailingMirrorIntegrityChecker] version of this function sends an email message instead.

The class [BenchmarkingMirrorIntegrityChecker]added two functions: preCheck and postCheck. This functions are called by the base class just before and just after the tests. These functions, though, are cooperative. While technically they override the super class functions of the same name, they actually invoke them in the same manner as typical Python _init_ functions. The [BenchmarkingMirrorIntegrityChecker]class starts a timer when preCheck is executed and then passes execution on to the preCheck function of its super class. Presumably, that class has its own initialization that needs to be done just before the checks are started. The postCheck function in [BenchmarkingMirrorIntegrityChecker], stops and logs the output of the timer. Again, it also passes the execution on to its superclass function of the same name.

This pattern of passing execution on to the superclass is repeated in several places in Sentry. When Sentry is actually checking the integrity of files, it takes actions based on the results of its tests. This is a list of the actions:

Action
fileActionGoodToBad
fileActionBadToGood
fileActionGoodNoChange
fileActionBadNoChange
mirrorActionGoodToBad
mirrorActionBadToGood
mirrorActionGoodNoChange
mirrorActionBadNoChange

Each of these actions corresponds to a function in these classes that uses the cooperative execution pattern. The base class [MirrorIntegrityChecker], these functions just make a string explaining what file or mirror is involved and what the action was. The string is added to a list called humanReadableActionList. When the checking of all mirrors is complete, the base class dumps this list to standard out. Of course, if the [EmailingMirrorIntegrityChecker] is in the inheritance chain, the list is emailed instead.

It's not very useful to just have a list of actions printed, we want something to actually happen according to the results of the tests. That's where the [DatabaseSourcedMirrorIntegrityChecker]comes in. This class has is own version of the Action functions listed in the table above. These functions actually mark the given file or mirror good or bad based on the Action in the database. This is what stops or starts a file from being served by Bouncer. Because these functions use the cooperative pattern, once they've done their database updates, they pass control to the super class, which, in this case, logs the action that was just taken to the humanReadableActionList.

If the [HistoricalMirrorIntegrityChecker]is mixed into the class hierarchy, it logs the actions to special tables within the database. It has its own versions of the action functions that behave cooperatively with all the other versions of these functions within the hierarchy.