Configuration parameters – In database or in config files?

Every application either accicentaly or by design, ends up having some system configuration parameters in the database and some in the configuration ( .ini or .xml or .config) files. But where are they better placed?

Lets look at config files:

• If the application uses databases, the database to connect to must be there in the config files, so this cannot be eliminated.
• There are some environment properties (outside our applications) which the runtime reads from the config files. they cannot be moved to database as well.
• Its usually easy to edit config files using any text editor.
• Its easier for development as it doesnot require any infrastructural code to read these parameters from the database.
• It is easy to distribute these.
• In the case of clustered application, the same parameters need to be synchronised to all nodes in the cluster.
• There are chances of making a mistake while editing it.
• In case there are multiple environments, maintaining different files with most common, but some different values is cumbersome.

Now lets look at databases.

• In case of changing a parameter in a cluster – we need to do that only once. 
• We need to write a small piece of code to efficiently load these values from the database, and keep them in memory for future use.
• We could allow for dynamic update of these parameters, but there would be a lot more code if this needs to be cached and all.
• We can provide an interface for users to change these.
• These parameters are easier to change and maintain as one change reflect in all machines.
•  In case of multiple environments, we could keep the parameters on these servers, and these parameters will not be affected when a new application deployment happens.
• For adding new parameters, SQL scripts need to be created for all environments.

So what does it mean? Which way to go?

In my experience, I prefer to keep as much in the database as possible, leaving in the config files only those which are inevitable, like the Database connection string, session parameters etc.

I also tend to have a simple application which can be used to maintain any set of name value pairs – which we use for editing the database.

How to be an architect?

I have answered asked and deliberated this quite a few times, and I am trying to put together my thought on it. I hope this puts some sense into this loaded word.

IMHO, an architect brings to the table an ability to make or assist technology and platform choices which let business achieve its core objectives.

Apart from this, an architect is also expected to understand and communicate to different stakeholders ( be it ISP who you are supposed to chose the bandwidth package on behalf of the client, be it OOAD expert modeling the system, be it project sponsor who has to make choices and compromises).

Deliverable of an architect is a partitioned view of the system, with technologies / products and other basis or constraints of these partitions.

Let me elaborate these points by examples.

Sometime back, a manager at a forward looking financial institution got a brilliant idea for a new product / service the bank could sell. The management gave her a small budget and a time of 6 months to prove that the concept works, it sells and is profitable by doing a pilot. So when she approaches an architect, the architect needs to be able to understand ( possibly by questioning) that the application needs to be low budget/low time. Scalability and maintainability and other goodnesses could take a back seat here. The architect needs to explain to the Business owner that – in case the product is successful, the application will require to be re-built before it could be put to a bigger use. In this case a so called poorly designed application is the best architecture. The Architect also needs to be aware of ( or need to hunt for) any COTS ( commercial off the shelp product) which could fit the need and the budget, instead of proposing a custom development.

Now lets take a second scenario. Lets say a Startup approaches an Architect. One of the business objectives of the Startup is to maximize the capital value of its IT investment, as it is likely that they would like to be acquired. Now as an architect, you are expected to propose leading edge technology which creates a product better than what is out there in the market. which covers all the basic definition of a good software of scalability, maintainibility, security, availability and the works. It should possibly avoid product and components which restrict the sale of the same or make it prohibitively expensive to scale.

Now lets take a case of a product which operates as a software factory – having a common code base but certain client specific customizations which are also to be maintained. In this case, testability and maintainbility become important.

Now coming to HOW does one become an architect?

An architech needs to be able to do rough estimations – so that a proper granularity of partitions could be achieved.

An architect needs to be able to gather Non Functional Requirements  and apply them ( like the examples above)

An architect needs to have or to be able to get opinion on different technologies and product options (whether they are mature, whether they are used, is support available, which products are available, which are popular). This usually comes from networking, googling, experimenting and /or by sending out RFI to the vendors.

An architect needs to have a background of application design, so that he/she is able to convert the partitions and their responsibility definition into concrete specifications and interface contracts – possibly with the help of application design experts.

An architect needs to have some background on performance tuning and application load testing.

An architect needs to have understanding of different software and hardware ways to enable application scalability and availability, and cost / compromises required for each of the options.

So in short:

Get involved in “technical” and non functional requirements gathering.

Get involved in deployment planning and deployment (and possibly on-site application support) and figure out the concerns datacenter support teams have and their stake and interest in the applications.

Get involved in estimation and figure out how an application development is broken into individual tasks which can be expecuted.

Get involved in design.

Get involved in performance testing and testing of non functional requirements.

If you are reading this, its likely that you are already doing one or all of the above. If so, probably you already are an architech and all you need to do is to figure out nice document templates to put content in.

I recommend two places to look at: http://www.opengroup.org/togaf/ and http://www.rgoarchitects.com/saf/
The latte, being created by a single person , is more practical.

Packaging – what developers dont do while creating installables.

Did you know that there is an industry out there which specializes in creating installers in which silent install works- using the installers provided by developers? I was blissfully ignorant about it, though designing an installer has been a part of my job description for ages.

An example of such a product is http://www.altiris.com/Products/Packaging.aspx

So what is wrong with the default installers?

A lot happens at conferences.. CMF2006

Historically, I have been sceptic of conferences.  Some very big software conferences usually have good sessions with very little opportunity for meaningful networking. Some are highly glorified sales pitches and product launches. CMF2006 on the contrary was a very cosy environment.

Some said that it was the “lack of clients”  which made the environment so friendly. But, I think it was the Danish hospitality which made it so.

Most people I met, were real people with real problems and real opinions – “practitioners”.

Many good people were present. There were a few things which did surprise me – size of the many vendors present there – rather the lack of it.

Companies under 10 million usd turnover were exhibiting, and were mainstream players in the nordic markets. If the amount of innovation that comes out of nordic region impresses you as much as it does me, you would sit up and take notice.

The one that impressed me is sitecore. Its .NET – that doesnt mean anything till you are a microsoft shop looking for MS based solutions. Chances are that CMS is the only Java solution that you have.

Among the people who impressed me most – the one who stands out is Eric Hartman . He has a very friendly “non threatning” approach. His session had real take aways. a gem of a person.

The biggest disappointment was consultant stressing over and over again that the makrt is so immature. I think whoever says that is wrong. Its a function of processing power and some unreasonble licensing prices which forces many to use sub-optimal options.

Just think about it – we can now do full text search in the blink of an eye – and it tolerates mis-spelt words as well. Categorization algorithm (bayesian) had always been available, but never more popular than now – thanks to email spam filtering. Disk space is now actually affordable, and network is cheap enough to make keeping flies at a central location viable.

Content input, search and dissemination is now a commodity.  There are a few problems of course. There is still some element of presentation which people like to put in the content and people dont agree on the minimal subset of formatting required during content authoring. I tend to go with subset allowed in Wiki notation  to be a part of the content.

Publishing to web is so easy now – look at Joomla which does what we want how we want, and we realize how hard the commercial product vendors have to work to compete with it.

The industry is now talking about the next phase, including managing “document components” instead of the whole.

There were two other talks which I am not sure I liked.

The “consultants” claiming that techies have landed us in this mess. This is something which I strongly disagree with. Typically “techie” led projects tend to implement what customers are asking for, so if you ask for Junk, you get junk out. But most software vendors, including my employer ( MindTree) dont work like that. We have Business Analysts who question, analyse and get a healthy brain storming session going when it comes to requirements. They make the requirement gathering time more productive. Of course there is a different way as well. Some clients want to go with the best practices followed in the industry. This is where many product vendors have now started offering “solutions” lets say for legal firms etc.

The second thing which I didnt like was that people tend to agree that many CMS projects fail. Eric made a statement saying that definition of failure should be relaxed to say – if the system is not used, it is failed. Currently agreed definitions include cost and time overruns as well. In my opinion when these projects are considered “failed”, it is usually because of inexperience of the users defining the requirements, or because the leading edge technology doesnt live up to its promise. Actually inexperience is a wrong word, users expect too much or too little from technology. If this is the first CMS project your organization is having, I tend to advocate a prototyping phase, let the users pilot it for a few months, and then finalize on the requirements. Hiring a consultant who tells you exactly you should do it is also a good approach. Ultimately, what works best for your organization depends on a lot of factors and its very likely that it will be an iterative process.

I recently relocated to india and had to go for customs clearence of household goods. They have a fully computerised process, but what computers have done is to replace hand-written forms with typed forms and to replace the clerk who used to carry the form around with a “send to” button. You still have to queue in front of 5 different offices for getting the 5 different steps of workflow done. This is also as common in the CMS world where we replace the manual workflow 1-1 with automated workflows, with all the goods and the bads coming with it.

Coming back to CMF – I think it was a good one as all practitioners realized that others faced the same problem as they did. They had a chance to ask for opinions – and chances were that they would get both sides of the coin. And it wasnt all about shameless marketing.