Since the earliest days of business computing, the
goal has been to identify solid, structured activities and automate them. As
a result, the first business processes to be automated were well-structured
redundant tasks such as payroll processing.
As the decades passed, information systems became more
sophisticated at capturing and deploying human intelligence within computer
systems, and now we see these types of systems:
-
Expert Systems - These online systems capture a
structured task and mimic human processing. An expert system makes
the decision without the aid of any human intuition. An example
would be Mycin, a system that applies physician intelligence in
analyzing blood samples.
-
Decision Support Systems - A DSS is a computerized
system that recognizes that human intuition is difficult to quantify and
automate. In a DSS, the human makes the decision, guided by
software that automates the more structured aspects of the problem
domain.
The line between an expert system and a decision
support system blurs in some cases when what is thought to be an intuitive
process is actually a well-structured problem with extremely complex
decision rules.
In one
notable case, a major soup manufacturer was about to loose a long-term
employee of forty years, who knew every intricacy of the tricky soup vats
within the company.
Initially
setting out to create a DSS, the decision analyst quizzed the employee over
a period of months and discovered that what was once thought to be intuition
was actually the application of a large set of well structured decision
rules. When this soup vat expert would say something like "I have a
feeling that the problem is X", it appeared to be human intuition to
those less knowledgeable observers.
However in
reality it was the application of a long forgotten decision rule or an
experiential case for which the individual had since lost conscious
knowledge. The application of the decision support system technology
eventually led to an expert system. This allowed the forty year worker
to retire comfortably, with the knowledge that all of his years of decision
rules had in fact been quantified, helping the soup company carry on without
him making even faster and better decisions as a whole.
A knowledge engineering system for Oracle data cleansing
If we start by examining known data errors to find
common patterns, a qualified software engineer can design Oracle-based
programs to detect these types of errors and quickly clean-up a large amount
of transposition errors, and successively refine the model to identify less
obvious data anomalies. We can also search for statistical "outliers",
data that violates the norms of the database as-a-whole.
By using well-understood best practices for Oracle data
cleansing a robust and flexible system can be created to dramatically reduce
data anomalies. Using an iterative cycles of refining the decision
rules, the DSS evolves to become increasingly accurate and powerful.
This is a DSS for Oracle data cleansing in a nutshell.
Note that we start by examining the "nature" of known data errors and seek
"fishy" data (statistically valid outliners) for creating the suggestion
lists for the human expert (the DQO).
The DQO then manually resolved the errors and works
with the DBA to refine the decision rules until they are 100% complete and
accurate using the "feedback loop" of successive rule refinement. At
that point, that component of the Oracle data cleansing is automated,
becoming an "expert system" component of the DSS.
| For expert Oracle data cleansing support and data
scrubbing consulting, use an expert from BC. We understand the
powerful Oracle data unification tools, and we can aid in improving
the data quality of any Oracle database, large or small. |
 |
References:
and include the URL for the page.