Flow of Database

EVOLUTION OF DATA SYSTEMS

         How were companies running their business before computers came into use? Even

at that time, organizations needed information to execute the business processes,

sell goods and services, and satisfy the needs of customers. Manual files supported

business operations. Accounting personnel performed manual calculations and prepared 

invoices. Payroll departments manually wrote the checks. Business operations

were reasonably satisfactory.

So, what happened? How did we get to the computer database systems of today?

When computers were introduced in the 1960s, computer file systems replaced the

manual files. This marked a significant leap in the way data was stored and retrieved

for business operations. What has been really happening from that time until now,

when database systems have become the norm? What prompted the progress

toward database systems?

         From the 1970s onward, two striking and remarkable phenomena were distinctly

observed. Refer to Figure 1-1 indicating these two major developments.

First, demand for information has escalated in every organization. Organizations

have steadily become global and widespread. Organizations have to contend with

fierce competitive pressures. They need vast and complex information to stay in

business and make a profit. Second, the past three decades have witnessed a huge,

explosive growth in information technology. Processors have become faster,

cheaper, and smaller. Operating systems have become powerful and robust. Data

storage media have expanded tremendously in capacity; data storage prices have

tumbled. Network and communication technology can now connect any remote site

without difficulty. Application programming and people-machine interface have

dramatically improved.

          The escalating demand for information and the explosive growth in information

technology have worked hand in hand to bring about the evolution to database

systems. Ever-increasing demand for information drives the need for better methods

of storing and retrieving data, for faster ways of processing data, and for improved

methods of providing information. The demand for more and better information

drove the technology growth. Progress in technology, in turn, spurred the capability 

to provide different types of information, not just to run day-to-day operations

of an organization, but also to make strategic decisions.

Let us first examine the pertinent aspects of the technology explosion as related

to data systems, because these are what we are specifically interested in. Then let

us discuss the escalating demand for information that has prompted better and

improved data systems.

          TECHNOLOGY EXPLOSION

If you have been in the information technology area for 5–10 years, you are certainly 

an eyewitness to the explosive growth. Growth is not confined to any one all aspects 

of the technology have been improving tremendously. Here aresome specifics:

• Twenty-five years ago, there were only 50,000 computers in the whole world;

now more than 500,000 are installed every day.

• More than 60% of American households have at least one computer; more than

50% have e-mail and Internet access.

• Growth of the Internet and the use of the Web have overshadowed the PC

breakthrough of the 1970s; at the beginning of 2000, about 50 million households

 worldwide were estimated to be using the Internet; by the end of 2005,

this number is expected to grow 10-fold.

• About 7 years ago, there were only 50 websites; now 100,000 are added every

hour.

• Databases in the terabyte range are becoming common; a few years ago, even

the gigabyte range was unusual.

• In the mid-1960s, programmers in large corporations had to write programs that

had to run on 12K machines; today even your personal computer at home has

10,000 times larger memory.

Growth has not been isolated here and there in hardware and software. We notice

explosive growth in all sectors of information technology. Let us proceed further to

look at specific areas of information technology that are related to data systems.

COMPUTER APPLICATIONS  

          Over the years, the types of computer applications have

changed and progressed from mere bookkeeping applications to multimedia and

data mining applications. Some of you might remember the days when the computer 

department was known as the data processing department. Applications in

those days just processed data in elementary ways to produce some reports. The

technology explosion resulted in a grand transition of computer usage from simple

to increasing sophistication. Review the following details.

Data Processing Applications (DP). In the early days of computing, computer

departments built applications just to replace clerical labor. Mostly, these applications

 performed simple accounting and financial functions. These applications produced

 straightforward reports. Speed and accuracy of the computer in performing

calculations were the primary factors. Computer systems stored and retrieved data

from magnetic tapes and earlier versions of disk drives. Applications used sequential 

or flat files to organize data.

Management Information Systems (MIS). In the next stage, growth of technology

manifested itself in applications that went beyond accounting and finance to 

supporting the entire core business of an organization. Applications began to appear

to process orders, manage inventory, bill customers, pay employees, and so on.

Organizations depended on their management information systems for their day-to-day

business. Storage and retrieval of data mostly depended on hard disks. Many 

applications adopted the use of database technology.

Decision-Support Systems (DSS). Further technology growth in processor speed,

storage media, systems software, and database techniques pushed the application

types to systems that supported strategic decision making. These applications 

are not meant for supporting day-to-day operations of a business but for providing

information to executives and managers to make strategic decisions. In which

markets should the company expand? Where should the next distribution warehouse 

be built? Which product lines should be discontinued? Which ones should be

boosted? These applications dealt with sales analysis, profitability analysis, and customer 

support. Decision-support systems made use of improved storage facilities

and newer features of database technology.

Data Warehousing (DW) and Data Mining (DM) Systems. In recent years, with the

enormous progress in processor scalability, mass storage, and database methods,

organizations are able to forge ahead with their applications, especially in building

data warehousing and data mining systems. These recent decision-support systems,

much more sophisticated than earlier attempts, require large volumes of data and

complex analytical techniques. These systems need large databases specially

designed and built separately from the databases that support the day-to-day operational systems.

DATA SYSTEMS

      What is the effect of the technology explosion on the way data is

organized? Over the years, how were businesses organizing data? We just looked at

the way applications have progressed from simpler types toward increasing sophistication. 

What about data systems? 

       Manual-Type Records. Very early computer applications worked with data stored

on punched cards and paper tapes. Keypunch operators prepared data on these

primitive media from manual files and records. Computer applications read data

from cards and tapes to prepare reports.

Sequential Files. Improved storage media such as magnetic tapes and early disk

drives enabled application developers to organize data as sequential (or flat) files.

      Each file contained data records of the same type arranged sequentially one after

the other, usually in the order in which they were created. Sorting techniques

allowed data records to be resorted in a different sequence.

Databases. Increased sophistication in data storage techniques on hard disk drives

and enhancements to operating systems enabled random and quick access of data.

    Data systems moved to a wholly new level. Applications were able to store data in

databases and retrieve data sequentially and randomly.

DEMAND FOR INFORMATION

    Of the two major factors that mutually contributed to the database approach to

computing, so far we have considered the explosive growth of technology. Let us

now turn our attention to the other factor, namely, the escalating demand for information. 

It is not just more information that organizations need. The demand for

information includes several dimensions.

   Consider how billing requirements and sales analysis have changed. In the early

years of computing, organizations were happy if they could bill their customers once

a month and review total sales by product quarterly. Now it is completely different.

Organizations must bill every sale right away to keep up the cash flow. They need

up-to-date customer balance and daily and cumulative sales totals by products. What

about inventory reconciliation? Earlier systems provided reports to reconcile inventory

 or to determine profitability only at the end of each month. Now organizations

need daily inventory reconciliation to manage inventory better, daily profitability

analysis to plan sales campaigns, and daily customer information to improve customer service.

In the earlier period of computing, organizations were satisfied with information

showing only current activity. They could use the information to manage day-to-day

business and make operational decisions. In the changed business climate of 

globalization and fierce competition, this type of information alone is no longer 

adequate. Companies need information to plan and shape their future. They need

information, not just to run day-to-day operations, but to make strategic decisions

as well.

     What about the delivery of information now compared to the early days of computing? 

Today, online information is the norm for most companies. Fast response

times and access to large volumes of data have become essential. Earlier computer

systems just provided reports, mostly once a month, a few once a week, and a small

number once a day.

      Organizations have come to realize that information is a key asset to be carefully managed 

and used for greater profitability. In summary, demand for information by today’s enterprises 

contains the following attributes:

• More information

• Newer purposes

• Different information types

• Integrated information

• Information to be shared 

• Faster access to information

WHY DATABASE SYSTEMS?

       We traced the evolution of data systems. We grasped the essentials of the explosive

growth of information technology. We noted the escalating demand of organizations

for information. We observed how growth in information technology and the

increased demand for information worked hand in hand. Increasing demand for

information spurred the growth of information technology. Growth of information technology, 

in turn, enabled organizations to satisfy the increasing demand for

information.

       Let us summarize the driving forces for organizations to adopt database systems.

A major reason is the inadequacy of the earlier file-oriented data systems. We shall

review the limitations and see how database systems overcome the limitations and

provide significant benefits.

The Driving Forces

      Among others, four major forces drove organizations to adopt database systems.

Information as a Corporate Asset. Today, companies strongly realize that information 

is a corporate asset similar to other assets such as cash, plant and equipment, 

or inventory. Proper management of key assets is essential for success.

Companies understand that it is essential to manage information as a key asset.

They understand the need to find improved methods for storing, retrieving, and

using information.

     Explosive Growth of Computer Technology. Computer technology, especially data

storage and retrieval systems, has grown in a phenomenal manner. Without growth

in this sector, it is unlikely that we could have progressed to database systems that

need sophisticated ways of data storage and retrieval.

Escalating Demand for Information. We have noted the increase in demand for

information by organizations, not only in volume but in the types of information as

well. If companies did not need more and newer types of information, there would

have been no impetus for development of database systems. The earlier data systems

might have been satisfactory.

Inadequacy of Earlier Data Systems. Suppose the earlier data systems were able

to meet the escalating demand for information. Then why bother to find better

methods? But the fact is that these earlier systems were grossly inadequate to meet

the information demands. Storage and management of large volumes of data were

not adequate. Finding and retrieving information were extremely difficult. Protecting 

the information asset of a company was nearly impossible with the earlier data

systems. Why was this so? How were the earlier systems inadequate? In what ways

could they not meet the information demands? Understanding the limitations will

give you a better appreciation for database systems.

DATABASE SYSTEMS MEET THE CHALLENGES

      As the demand for information escalated, it became urgent to overcome the 

limitations of file-oriented data systems. With these limitations, companies could not

meet the requirements of increased demand for information. They needed a different 

approach to storing, retrieving, and managing data. They could not afford 

the productivity losses. They could not waste space because of data duplication in

file-oriented systems.

       Specialists at Rockwell and General Electric began to work on better methods

for managing data. These methods attempted to overcome the limitations of 

file-oriented systems. Data and processing logic had to be separated so as to 

improve programmer productivity. The new approach of using databases instead of

conventional flat files addressed the challenges for meeting the increased demand

for information. The database approach overcame the limitations of the earlier data

systems and produced enormous benefits. Let us review the specific benefits and

understand in what way the database approach is superior to the earlier data

systems.

       Minimal Data Redundancy Unlike file-oriented data systems where data 

are duplicated among various applications, database systems integrate all the 

data into one logical structure. Duplication of data is minimized. Wastage of 

storage space is eliminated. Going back to the bank example, with a database,

customer data is not duplicated in the checking account, savings account, and loan

account applications. Customer data is entered and maintained in only one place in

the database.

     Sometimes, in a database, a few data elements may have to be duplicated. Let us

say that product data consist of product number, description, price, and the 

corresponding product line number. All the fields relating to product line data are kept

separately. Whenever the details of products and product lines are needed in 

applications, both data structures are retrieved from the database. Suppose a heavily used

product forecast application needs all the details of the product from product data

and just the product line description from the product line data. In that case, it will

be efficient for the product data to duplicate the product line description from the

product line data. Thus, in some instances, data duplication is permitted in a 

database for the purpose of access efficiency and performance improvement. However,

such data duplications are kept to a minimum.

      Data integrity in a database means reduction of data inconsistency.

Because of the elimination or control of data redundancy, a database is less prone

to errors creeping in through data duplication. Field sizes and field formats are the

same for all applications. Each application uses the same data from one place in the

database. In a bank, names and addresses will be the same for checking account,

savings account, and loan applications.

      Data Integration In a database, data objects are organized into single logical data

structures. For example, in file-oriented data systems, data about employees are 

scattered among the various applications. The payroll application contains employee

name and address, social security number, salary rate, deductions, and so on. The

pension plan application contains pension data about each employee, whereas the

human resources application contains employee qualifications, skills, training, and

education. However, all data about each employee are integrated and stored

together in a database.

So, in a database, data about each business object are integrated and stored separately 

as customer, order, product, invoice, manufacturer, sale, and so on. Data integration 

enables users to understand the data and the relationships among data

structures easily. Programmers needing data about a business object can go to one

place to get the details. For example, data about orders are consolidated in one place

as order data.

       Data Sharing This benefit of database systems follows from data integration. The

various departments in any enterprise need to share the company’s data for proper

functioning. The sales department needs to share the data generated by the accounting 

department through the billing application. Consider the customer service

department. It needs to share the data generated by several applications. The customer

 service application needs information about customers, their orders, billings,

payments, and credit ratings. With data integration in a database, the application

can get data from distinct and consolidated data structures relating to customer,

orders, invoices, payments, and credit status.

Data sharing is a major benefit of database systems. Each department shares the

data in the database that are most pertinent to it. Departments may be interested

in data structures as follows:

Sales department—Customer/Order

Accounting department—Customer/Order/Invoice/Payment

Order processing department—Customer/Product/Order

Inventory control department—Product/Order/Stock Quantity/Back Order

Quantity

Database technology lets each application use the portion of the database that

is needed for that application. User views of the database are defined and controlled. 

We will have more to say about user views in later chapters.

     Uniform Standards We have seen that, because of the spread of duplicate data

across applications in file-oriented data systems, standards cannot be enforced easily

and completely. Database systems remove this difficulty. As data duplication is 

controlled in database systems and as data is consolidated and integrated, standards

can be implemented more easily. Restrictions and business rules for a single data

element need to be applied in only one place. In database systems, it is possible to

eliminate problems from homonyms and synonyms.

    Security Controls Information is a corporate asset and, therefore, must be 

protected through proper security controls. In file-oriented systems, security controls

cannot be established easily. Imagine the data administrator wanting to restrict and

control the use of data relating to employees. In file-oriented systems, control has

to be exercised in all applications having separate employee files. However, in a

database system, all data about employees are consolidated, integrated, and kept in

one place. Security controls on employee data need to be applied in only one place

in the database. Database systems make centralized security controls possible. It is

also easy to apply data access authorizations at various levels of data.

      Data Independence Remember the lack of data independence in file-oriented

systems where computer programs have data structure definitions embedded within

the programs themselves. In database systems, file or data definitions are separated

out of the programs and kept within the database itself. Program logic and data

structure definitions are not intricately bound together. In a client/server environment, 

data and descriptions of data structures reside on the database server, whereas

the code for application logic executes on the client machine or on a separate application server.

    Reduced Program Maintenance This benefit of database systems results primarily

 from data independence in applications. If the customer data structure

changes by the addition of a field for cellular phone numbers, then this change is

made in only one place within the database itself. Only those programs that need

the new field need to be modified and recompiled to make use of the added piece

of data. Within limits, you can change programs or data independently.

    Simpler Backup and Recovery In a database system, generally all data are in

one place. Therefore, it becomes easy to establish procedures to back up data. All

the relationships among the data structures are also in one place. The arrangement

of data in database systems makes it easier not only for backing up the data but

also for initiating procedures for recovery of data lost because of malfunctions.

(Article from

Database Design and Development: An Essential Guide for IT Professionals by Paulraj Ponniah
ISBN 0-471-21877-4 Copyright © 2003 by John Wiley and Sons, Inc)