VIDEO
EVOLUTION OF DATA SYSTEMS
How were companies running their business before computers came into use? Even
at that time, organizations needed information to execute the business processes,
sell goods and services, and satisfy the needs of customers. Manual files supported
business operations. Accounting personnel performed manual calculations and prepared
invoices. Payroll departments manually wrote the checks. Business operations
were reasonably satisfactory.
So, what happened? How did we get to the computer database systems of today?
When computers were introduced in the 1960s, computer file systems replaced the
manual files. This marked a significant leap in the way data was stored and retrieved
for business operations. What has been really happening from that time until now,
when database systems have become the norm? What prompted the progress
toward database systems?
From the 1970s onward, two striking and remarkable phenomena were distinctly
observed. Refer to Figure 1-1 indicating these two major developments.
First, demand for information has escalated in every organization. Organizations
have steadily become global and widespread. Organizations have to contend with
fierce competitive pressures. They need vast and complex information to stay in
business and make a profit. Second, the past three decades have witnessed a huge,
explosive growth in information technology. Processors have become faster,
cheaper, and smaller. Operating systems have become powerful and robust. Data
storage media have expanded tremendously in capacity; data storage prices have
tumbled. Network and communication technology can now connect any remote site
without difficulty. Application programming and people-machine interface have
dramatically improved.
The escalating demand for information and the explosive growth in information
technology have worked hand in hand to bring about the evolution to database
systems. Ever-increasing demand for information drives the need for better methods
of storing and retrieving data, for faster ways of processing data, and for improved
methods of providing information. The demand for more and better information
drove the technology growth. Progress in technology, in turn, spurred the capability
to provide different types of information, not just to run day-to-day operations
of an organization, but also to make strategic decisions.
Let us first examine the pertinent aspects of the technology explosion as related
to data systems, because these are what we are specifically interested in. Then let
us discuss the escalating demand for information that has prompted better and
improved data systems.
TECHNOLOGY EXPLOSION
If you have been in the information technology area for 5–10 years, you are certainly
an eyewitness to the explosive growth. Growth is not confined to any one all aspects
of the technology have been improving tremendously. Here aresome specifics:
• Twenty-five years ago, there were only 50,000 computers in the whole world;
now more than 500,000 are installed every day.
• More than 60% of American households have at least one computer; more than
50% have e-mail and Internet access.
• Growth of the Internet and the use of the Web have overshadowed the PC
breakthrough of the 1970s; at the beginning of 2000, about 50 million households
worldwide were estimated to be using the Internet; by the end of 2005,
this number is expected to grow 10-fold.
• About 7 years ago, there were only 50 websites; now 100,000 are added every
hour.
• Databases in the terabyte range are becoming common; a few years ago, even
the gigabyte range was unusual.
• In the mid-1960s, programmers in large corporations had to write programs that
had to run on 12K machines; today even your personal computer at home has
10,000 times larger memory.
Growth has not been isolated here and there in hardware and software. We notice
explosive growth in all sectors of information technology. Let us proceed further to
look at specific areas of information technology that are related to data systems.
COMPUTER APPLICATIONS
Over the years, the types of computer applications have
changed and progressed from mere bookkeeping applications to multimedia and
data mining applications. Some of you might remember the days when the computer
department was known as the data processing department. Applications in
those days just processed data in elementary ways to produce some reports. The
technology explosion resulted in a grand transition of computer usage from simple
to increasing sophistication. Review the following details.
Data Processing Applications (DP) . In the early days of computing, computer
departments built applications just to replace clerical labor. Mostly, these applications
performed simple accounting and financial functions. These applications produced
straightforward reports. Speed and accuracy of the computer in performing
calculations were the primary factors. Computer systems stored and retrieved data
from magnetic tapes and earlier versions of disk drives. Applications used sequential
or flat files to organize data.
Management Information Systems (MIS) . In the next stage, growth of technology
manifested itself in applications that went beyond accounting and finance to
supporting the entire core business of an organization. Applications began to appear
to process orders, manage inventory, bill customers, pay employees, and so on.
Organizations depended on their management information systems for their day-to-day
business. Storage and retrieval of data mostly depended on hard disks. Many
applications adopted the use of database technology.
Decision-Support Systems (DSS) . Further technology growth in processor speed,
storage media, systems software, and database techniques pushed the application
types to systems that supported strategic decision making. These applications
are not meant for supporting day-to-day operations of a business but for providing
information to executives and managers to make strategic decisions. In which
markets should the company expand? Where should the next distribution warehouse
be built? Which product lines should be discontinued? Which ones should be
boosted? These applications dealt with sales analysis, profitability analysis, and customer
support. Decision-support systems made use of improved storage facilities
and newer features of database technology.
Data Warehousing (DW) and Data Mining (DM) Systems . In recent years, with the
enormous progress in processor scalability, mass storage, and database methods,
organizations are able to forge ahead with their applications, especially in building
data warehousing and data mining systems. These recent decision-support systems,
much more sophisticated than earlier attempts, require large volumes of data and
complex analytical techniques. These systems need large databases specially
designed and built separately from the databases that support the day-to-day operational systems.
DATA SYSTEMS
What is the effect of the technology explosion on the way data is
organized? Over the years, how were businesses organizing data? We just looked at
the way applications have progressed from simpler types toward increasing sophistication.
What about data systems?
Manual-Type Records. Very early computer applications worked with data stored
on punched cards and paper tapes. Keypunch operators prepared data on these
primitive media from manual files and records. Computer applications read data
from cards and tapes to prepare reports.
Sequential Files. Improved storage media such as magnetic tapes and early disk
drives enabled application developers to organize data as sequential (or flat) files.
Each file contained data records of the same type arranged sequentially one after
the other, usually in the order in which they were created. Sorting techniques
allowed data records to be resorted in a different sequence.
Databases. Increased sophistication in data storage techniques on hard disk drives
and enhancements to operating systems enabled random and quick access of data.
Data systems moved to a wholly new level. Applications were able to store data in
databases and retrieve data sequentially and randomly.
DEMAND FOR INFORMATION
Of the two major factors that mutually contributed to the database approach to
computing, so far we have considered the explosive growth of technology. Let us
now turn our attention to the other factor, namely, the escalating demand for information.
It is not just more information that organizations need. The demand for
information includes several dimensions.
Consider how billing requirements and sales analysis have changed. In the early
years of computing, organizations were happy if they could bill their customers once
a month and review total sales by product quarterly. Now it is completely different.
Organizations must bill every sale right away to keep up the cash flow. They need
up-to-date customer balance and daily and cumulative sales totals by products. What
about inventory reconciliation? Earlier systems provided reports to reconcile inventory
or to determine profitability only at the end of each month. Now organizations
need daily inventory reconciliation to manage inventory better, daily profitability
analysis to plan sales campaigns, and daily customer information to improve customer service.
In the earlier period of computing, organizations were satisfied with information
showing only current activity. They could use the information to manage day-to-day
business and make operational decisions. In the changed business climate of
globalization and fierce competition, this type of information alone is no longer
adequate. Companies need information to plan and shape their future. They need
information, not just to run day-to-day operations, but to make strategic decisions
as well.
What about the delivery of information now compared to the early days of computing?
Today, online information is the norm for most companies. Fast response
times and access to large volumes of data have become essential. Earlier computer
systems just provided reports, mostly once a month, a few once a week, and a small
number once a day.
Organizations have come to realize that information is a key asset to be carefully managed
and used for greater profitability. In summary, demand for information by today’s enterprises
contains the following attributes:
• More information
• Newer purposes
• Different information types
• Integrated information
• Information to be shared
• Faster access to information
WHY DATABASE SYSTEMS?
We traced the evolution of data systems. We grasped the essentials of the explosive
growth of information technology. We noted the escalating demand of organizations
for information. We observed how growth in information technology and the
increased demand for information worked hand in hand. Increasing demand for
information spurred the growth of information technology. Growth of information technology,
in turn, enabled organizations to satisfy the increasing demand for
information.
Let us summarize the driving forces for organizations to adopt database systems.
A major reason is the inadequacy of the earlier file-oriented data systems. We shall
review the limitations and see how database systems overcome the limitations and
provide significant benefits.
The Driving Forces
Among others, four major forces drove organizations to adopt database systems.
Information as a Corporate Asset. Today, companies strongly realize that information
is a corporate asset similar to other assets such as cash, plant and equipment,
or inventory. Proper management of key assets is essential for success.
Companies understand that it is essential to manage information as a key asset.
They understand the need to find improved methods for storing, retrieving, and
using information.
Explosive Growth of Computer Technology. Computer technology, especially data
storage and retrieval systems, has grown in a phenomenal manner. Without growth
in this sector, it is unlikely that we could have progressed to database systems that
need sophisticated ways of data storage and retrieval.
Escalating Demand for Information . We have noted the increase in demand for
information by organizations, not only in volume but in the types of information as
well. If companies did not need more and newer types of information, there would
have been no impetus for development of database systems. The earlier data systems
might have been satisfactory.
Inadequacy of Earlier Data Systems. Suppose the earlier data systems were able
to meet the escalating demand for information. Then why bother to find better
methods? But the fact is that these earlier systems were grossly inadequate to meet
the information demands. Storage and management of large volumes of data were
not adequate. Finding and retrieving information were extremely difficult. Protecting
the information asset of a company was nearly impossible with the earlier data
systems. Why was this so? How were the earlier systems inadequate? In what ways
could they not meet the information demands? Understanding the limitations will
give you a better appreciation for database systems.
DATABASE SYSTEMS MEET THE CHALLENGES
As the demand for information escalated, it became urgent to overcome the
limitations of file-oriented data systems. With these limitations, companies could not
meet the requirements of increased demand for information. They needed a different
approach to storing, retrieving, and managing data. They could not afford
the productivity losses. They could not waste space because of data duplication in
file-oriented systems.
Specialists at Rockwell and General Electric began to work on better methods
for managing data. These methods attempted to overcome the limitations of
file-oriented systems. Data and processing logic had to be separated so as to
improve programmer productivity. The new approach of using databases instead of
conventional flat files addressed the challenges for meeting the increased demand
for information. The database approach overcame the limitations of the earlier data
systems and produced enormous benefits. Let us review the specific benefits and
understand in what way the database approach is superior to the earlier data
systems.
Minimal Data Redundancy Unlike file-oriented data systems where data
are duplicated among various applications, database systems integrate all the
data into one logical structure. Duplication of data is minimized. Wastage of
storage space is eliminated. Going back to the bank example, with a database,
customer data is not duplicated in the checking account, savings account, and loan
account applications. Customer data is entered and maintained in only one place in
the database.
Sometimes, in a database, a few data elements may have to be duplicated. Let us
say that product data consist of product number, description, price, and the
corresponding product line number. All the fields relating to product line data are kept
separately. Whenever the details of products and product lines are needed in
applications, both data structures are retrieved from the database. Suppose a heavily used
product forecast application needs all the details of the product from product data
and just the product line description from the product line data. In that case, it will
be efficient for the product data to duplicate the product line description from the
product line data. Thus, in some instances, data duplication is permitted in a
database for the purpose of access efficiency and performance improvement. However,
such data duplications are kept to a minimum.
Data integrity in a database means reduction of data inconsistency.
Because of the elimination or control of data redundancy, a database is less prone
to errors creeping in through data duplication. Field sizes and field formats are the
same for all applications. Each application uses the same data from one place in the
database. In a bank, names and addresses will be the same for checking account,
savings account, and loan applications.
Data Integration In a database, data objects are organized into single logical data
structures. For example, in file-oriented data systems, data about employees are
scattered among the various applications. The payroll application contains employee
name and address, social security number, salary rate, deductions, and so on. The
pension plan application contains pension data about each employee, whereas the
human resources application contains employee qualifications, skills, training, and
education. However, all data about each employee are integrated and stored
together in a database.
So, in a database, data about each business object are integrated and stored separately
as customer, order, product, invoice, manufacturer, sale, and so on. Data integration
enables users to understand the data and the relationships among data
structures easily. Programmers needing data about a business object can go to one
place to get the details. For example, data about orders are consolidated in one place
as order data.
Data Sharing This benefit of database systems follows from data integration. The
various departments in any enterprise need to share the company’s data for proper
functioning. The sales department needs to share the data generated by the accounting
department through the billing application. Consider the customer service
department. It needs to share the data generated by several applications. The customer
service application needs information about customers, their orders, billings,
payments, and credit ratings. With data integration in a database, the application
can get data from distinct and consolidated data structures relating to customer,
orders, invoices, payments, and credit status.
Data sharing is a major benefit of database systems. Each department shares the
data in the database that are most pertinent to it. Departments may be interested
in data structures as follows:
Sales department—Customer/Order
Accounting department—Customer/Order/Invoice/Payment
Order processing department—Customer/Product/Order
Inventory control department—Product/Order/Stock Quantity/Back Order
Quantity
Database technology lets each application use the portion of the database that
is needed for that application. User views of the database are defined and controlled.
We will have more to say about user views in later chapters.
Uniform Standards We have seen that, because of the spread of duplicate data
across applications in file-oriented data systems, standards cannot be enforced easily
and completely. Database systems remove this difficulty. As data duplication is
controlled in database systems and as data is consolidated and integrated, standards
can be implemented more easily. Restrictions and business rules for a single data
element need to be applied in only one place. In database systems, it is possible to
eliminate problems from homonyms and synonyms.
Security Controls Information is a corporate asset and, therefore, must be
protected through proper security controls. In file-oriented systems, security controls
cannot be established easily. Imagine the data administrator wanting to restrict and
control the use of data relating to employees. In file-oriented systems, control has
to be exercised in all applications having separate employee files. However, in a
database system, all data about employees are consolidated, integrated, and kept in
one place. Security controls on employee data need to be applied in only one place
in the database. Database systems make centralized security controls possible. It is
also easy to apply data access authorizations at various levels of data.
Data Independence Remember the lack of data independence in file-oriented
systems where computer programs have data structure definitions embedded within
the programs themselves. In database systems, file or data definitions are separated
out of the programs and kept within the database itself. Program logic and data
structure definitions are not intricately bound together. In a client/server environment,
data and descriptions of data structures reside on the database server, whereas
the code for application logic executes on the client machine or on a separate application server.
Reduced Program Maintenance This benefit of database systems results primarily
from data independence in applications. If the customer data structure
changes by the addition of a field for cellular phone numbers, then this change is
made in only one place within the database itself. Only those programs that need
the new field need to be modified and recompiled to make use of the added piece
of data. Within limits, you can change programs or data independently.
Simpler Backup and Recovery In a database system, generally all data are in
one place. Therefore, it becomes easy to establish procedures to back up data. All
the relationships among the data structures are also in one place. The arrangement
of data in database systems makes it easier not only for backing up the data but
also for initiating procedures for recovery of data lost because of malfunctions.
