Maintain Overhaul - Modules and the Database - Maintain

Preface

Maintain is on track for a significant update. Not nessicarily a rewrite, but a serious switcheroo of the internal workings, specifically concerning dao's.

The Problem...

This story started with a simple question posed by cropj in the #Maintain irc channel. He was making a module that concerned one of the hosts' "optional" fields. Because these fields are not required, they are not validated, and so although there may be an optional field for a date, this field is really just a string and a user can put just about anything they want in there. This was unacceptable to cropj's needs, because his module requred there to be a date in that field, and because it was not validated/enforced, there was no way to tell what was going to come out from that field when it got into users hands.

When this problem was brought to the Maintain team, the first thought was to simply require the that these "optional" fields be validated properly. Unfortunately, some users of Maintain exploit these unvalidated fields in order to store data about objects that Maintain doesn't provide fields for. This is "a clever use of game mechanics" but nonetheless fundamentally flawed. Users should be able to add these kinds of extra fields if they want to store specific data about objects.

Futhermore, modules as they are right now have no easy way to create their own tables for their own use. So if a module wanted to store any kind of data for its own purposes, that would not be easy to code.

So we decided that it would be great if users could add these custom fields to objects that already exist in Maintain, such as the host object. (Most likely via a module) Furthermore, the module engine must allow for this, and must be altered also. When this was all said and done, we wanted to have a module written that would add these custom fields for users to the database. As a pre-requisite of this module, we must allow modules the ability to alter the database easily but safely, so that they may use it as module developers see fit.

But we have some other issues concerning these features...

The Sub-Problems

The problem is fully explained above, and we as a team have agreed that it is important for users to be able to add custom fields to objects as well as have modules take advantage of the database. There are some serious issues per each solution however...

Adding Extra Fields to Objects

Even if you could add a table to the database from a module (which you can't do easily), or add a field to an already existing table (which you also, basically, cannot do with any sort of ease), or write to and read from this new table/field (which you would have difficulty doing so in a useful way), the object in question will have no idea that this extra field exists.

When Maintain asks a host dbdo for all its information, the only way it knows what fields to get is because we have told the host where to get this information in its extended dao. So if you wanted to add new fields to any pre-existing object, you would have to touch and regenerate the dao completely. ewww...

Adding Extra Tables in the DB for Modules

But the aforementioned issue is further compunded by a modules' inability to create tables/fields in the database easily. And creating the table is just the first step, we must also provide a means for users to easily use these new tables/fields. Futhermore, if a users module adds a field to the host object, we need a way for the host to know about this addition, and return this data when asked. In theory, users should be able to make these new tables for their modules, essentially making new object all together. So in turn, these new objects should also be nice to other modules who want to extend them by adding extra fields. Double eww...

Overview

Maintain 3.0.0 is not broken in any sense of the word. These new features will be nice for module writers and users in the long run, and so we feel it is important to implement them. Furthermore, the solution we have come up with will force us to adapt our code to be more modular, which is always an excellent time investment.

The Potential Solutions

This is currently a work in progress; however, some ideas are currently under discussion.

Keep the DB_DataObjects in the Core of Maintain

This was the first proposed solution for the issue. While this issue may be easier to implement than the other solutions (though, after analysis, it did not appear to be that way) it does not appear to be the ideal solution. There are a few pros, but the cons outweigh the benefits in comparison to other the other currently proposed option.

Pros

New hooks can be added to extend current base objects.
Modules could somewhat be easily integratable into the core objects.

Cons

Adding your own object that is not in the core of maintain would be very difficult (for example, IP Phones)
We would have to provide many special cases in the core for customizability
Custom data types, if possible, would not be able to be further extended
This would make the migration to full modularity in maintain difficult

Make All Objects in Maintain Modular

While this option seems, at a first glance, to be more work than keeping DB_DataObjects in Maintain's core it is not necessarily going to be harder to implement. It definitely would make maintain much cleaner, and could provide an almost entirely modular core to Maintain. This means that Maintain's core, rather than holding several core object files, would be a series of module hooks. Instead of having core objects, the core objects would instead be core modules that contain the old core objects. This way, maintain could be easily made entirely modular, which is the goal. Also, since every object is a module, adding a new object would be as simple as writing a module.

Pros

Maintain will be almost entirely (if not completely entirely) modular.
Modules can be created that allow new objects to be added to Maintain (for example, IP phones)
Modules that add new objects to Maintain can be easily extended
Core objects can be easily extended
Maintain's core codebase would become cleaner and more easily extended.

Cons

Implementation may be very difficult and/or time consuming
New bugs may be introduced

Implementation

The implementation of this solution is still being worked out; the following is the current thoughts on how this should be implemented.

Module Table API

A framework would be created to make it simple for a user to add a table to maintain for each module. A simple function could be called during the _install() function, such as make_mod_table($sqlfile), or upgrade_mod_table($sqlfile) where $sqlfile is the path to a file that contains the sql that creates all the columns in the new table. During _uninstall() a function could be called that asks the user if they would like to remove the module's table, such as remove_mod_table().

Using DAO Modules

This is where things get to be a bit trickier. Normally, DBDOs have to be generated every time an object is changed. However, this doesn't work very well when you want to be able to extend an object. So, instead of generating a DBDO when it's changed in the core, you have a function that joins the new DBDO for the module to the old object's DBDO. Then, objects that just need all of the data can call the new DBDO, and all of the objects that just need the core data can call the old DBDO.

Class Structure

The new class structure is currently under discussion; however, the preliminary design is as follows:
Core

Module -> Core_Module -> Core_DAO -> host

Custom

Module -> DAO_Mod -> IP_Phone

DBDO Implementation

DBDOs will be implemented in such a way that they can be easily created and extended by modules. To do this, we can have multiple versions of each DBDO. The first version of the DBDO will be the original object, and all other versions of the DBDO will be an extended version of the DBDO. To do this, the joinAdd function can be used.

$dbdo1->joinAdd($dbdo2);

Functions can be created then to get access to the module $dbdo

function getModDBDO($object)
{
    //set $dbdo to dbdo of type $object
    //loop through all mods extending $dbdo, and join them
    return $dbdo;
}

If it were implemented this way, you could get quick access to the base object's fields (assuming you do not need any of the extended data) or you could easily get an extended object. This way all of the objects in maintain can be modularized; and all objects, including new objects, can be extended with more modules. If they are not extended, getModDBDO would just return the old value. In any situation in which all of the data needs to be collected (for example, a displayobject) you could just request for the new DBDO.