Think at these questions:
- Is my code faster/slower between version A and B?
- Is my code using much/more memory between version A and B?
- Is someone still using feature A?
- Which are the most used features?
One of the main stopper to do that is that is really difficult modify your application to add some stats support, because you really don't know what are your questions and you don't know what kind of output do you want.
What do you want is just a tiny call like: collect("func(x) time", 10sec)
And sometimes later you can decide.. ok I want to see what is the average of func(x) time between jan-feb (version A) and mar-apr (version B).
Or if you want keep track of features used you can call something like: collect("feature-used", "My Feature A"). And later you can decide to query for specified feature X to see when is last time that was used, or you can query for the most used features or something else.. but is really difficult to know in advance what you want to keep track.
Ok, now that you've understood a bit the problem that we want to solve, the fun part begins.
The main idea is to have a super light-weight "Stats Uploader" to collect your data with one single line of code and send to a collector, later on you can ask questions to your data (completely detached from your application).
The Aggregator fetch the data required to answer your question and applies your custom logic to extract your answer.
The Viewer is just a front-end to display nicely your data, like a web service that plot some charts and table. It ask questions to the aggregator and displays to you.
The code is available on github at https://github.com/matteobertozzi/skvoz.
Probably I'll give a talk about this at EuroPython (EP2012) .