Saturday, January 13, 2018

Database Design thoughts : A personal non-academic perspective.

The benefits of any database model, be it relational or non-relational is
  1. Data storage is efficient.
  2. Data retrival is efficient.
  3. Reporting is efficient
  4. Data transformation into Information is easy and self documenting the business / process. 
The first 3 benefits get a lot of focus while designing a database. In today's scenarios, implementations to achieve these benefits , are mainly managed by IT departments. The 4th benefit remains unspoken &  expected to be outcome of the first three.

4th benefit requires equal , if not more focus during the design phase. It should be the starting point of designing. Product managers and DBAs need to sit down and dream a little bit about current and future scenarios. After they have a rough draft of things to come, security & legal teams' input comes next in line. There are many academic papers and thoughts. 
         However after seeing difficult and messy designing of databases, The designing considerations boil down to :-
  1. What legal rules apply to the data being stored.
  2. What is data entry point.
  3. What is data exit point.
  4. At what stage or condition should the data be removed completely from the database.
  5. How to categorize data segments for security and privacy.
  6. What personnel and roles should have direct access to data.
  7. What will be data policy of the company and who will define the policy.
Each of the above , including legal, requires workflow with clear cut process in place. 
These aspects of designing used to be DBA's responsibility but in changing times when the role of DBA's is diminishing to implementing data storage, retrieval and reporting efficiency , Product managers and Security teams' involvement increases considerably.

Implementation of what comes out of the 4th Benefit falls on the shoulders of IT departments.  IT departments are at their best when it comes to choice of correct technology , machines and storage. They would also be able to analyze what type of database technology is best for the organization. Database technologies dictate datatypes, tables, columns, rows, roles, privileges and so on. Modularly designed database are low-maintenance. They also increase efficiency of developers who create in-house applications. Examples of modularity are:-
  1. Authentication module
  2. Authorization module
  3. Sales module 
  4. Marketing module
  5. Billing module 
  6. Decision making module (data warehouse module)
  7. and so on...

another module 'Link module' would be needed to to implement efficient storage and retrieval of data.  DFD should be also created and available as and when needed.