Watching the Watchers

It's time we added the term "privacy architecture" to our vocabularies

Ralph Kimball

I have been increasingly concerned in recent months over an ethical dilemma that I face in my data Webhouse consulting and education business regularly: privacy. I’m certain you, as designers and implementers of data warehouses, face that dilemma as well. We are building the infrastructure for effectively sharing information. We are tying far-flung databases together. We are analyzing customer behavior, evaluating credit risks, investment risks, and health risks. At the same time, we are building integrated, accessible data Webhouse profiles on most of our fellow citizens. When we talk about the technical details of “conforming the customer dimension,” it is too easy to focus on the minutiae and lose sight of the larger system we are building.

The issue of personal information privacy has become a national debate. Web users are alarmed at revelations of how the Web sites we visit collect personal dossiers on each of us. The Clinton-Gore administration has begun to push hard for new federal legislation that would substantially define and limit the sharing of personal information. On April 21 of this year, a very tough new set of privacy laws known as COPPA (Child’s On-Line Privacy Protection Act) went into effect. COPPA provides substantial penalties for Web sites that inappropriately gather personal information from minors.

To be honest, I am ambivalent about the role we play in all of this. There are powerful positive and negative arguments for gathering personal information. There is also a certain unstoppable momentum that data Webhouse professionals cannot ignore because of the fast pace of technology and the slow pace of legislative action. Even if the Clinton-Gore proposals turn out to be fundamentally reasonable, it will be at least two years before we’ll see any serious debate about this legislation, given election year realities.

I do not intend this column as a call-to-arms for a particular view. But I think we can make a number of predictions about the impact of the privacy debate on all of us, and it is time we added the term “privacy architecture” to our vocabularies.

Beneficial Uses and Insidious Abuses

In my opinion, the core of the privacy dilemma is the conflict between the beneficial uses and insidious abuses of personal information. We often allow corporations to gather our personal information when we only consider the beneficial uses. And we usually don’t understand or anticipate companies’ abuses of that same gathering of information when we approve it. Consider the following examples.

Personal medical information. The beneficial uses are obvious and compelling. We want our doctors to have complete information about us so that they can provide the most informed care. We recognize that insurance companies need access to our medical records so that they can reimburse the health care providers. Most of us agree that aggregated data about symptoms, diagnoses, treatments, and outcomes is valuable for society as a whole. Furthermore, we see the need to tie these medical records to fairly detailed demographic and behavioral information. Is the patient a smoker? How old is the patient? But the insidious abuses are nearly as riveting as the benefits. I don’t want my personal medical details to be available to anyone other than my doctor. I would prefer that the insurance claims-processing clerk did not see my name, but that is probably unrealistic. I certainly don’t want marketing- oriented third parties buying my personal medical information. I don’t want anyone discriminating against me because of my health status, age, or genetic predispositions.

Purchase behavior. Retailers’ beneficial uses of my purchase behavior let them give me personalized service. In fact, when I trust a retailer, I am quite happy to provide a customization profile listing my interests, if that focuses the choices down to a manageable number and alerts me to new products that might interest me. I want the retailer to know me well enough to handle questions, payment issues, delivery problems, and product returns in a low-stress way. But insidious abuses of my purchase behavior drive me ballistic. I do not wish any third party to solicit me through junk mail, email, or over the telephone; they also ignore my requests to opt-out.

Safety and security in public facilities. In this day and age, most of us are grateful for a feeling of security in airports, in front of bank teller machines, and in parking garages. We wish the people who deliberately run red traffic lights would stop endangering the rest of us. Most of us accept the presence of cameras and license plate recognition systems in these public places as an effective compromise that increases our safety and security. The legal system, which ultimately reflects our society’s values, has solidly supported the use of these kinds of surveillance technologies. But the insidious abuses of cameras and citizen-tracking systems are scary and controversial. We have the technical ability to create a national image database of every citizen and identify most of the faces that cross through airport security gates. How is the accumulated record of my travels going to be used, and by whom? Systems that track this type of information are being tested in Europe already. Ironically, a mandate from the U.S. Federal Aviation Administration that states that individual airports cannot introduce technology that allows them a “marketing advantage” related to safety is one of the main impediments to implementing such systems in the United States. So the entire U.S. airport system would have to adopt this technology all at once.

Who Owns Your Personal Data?

We all have a natural inclination to believe that we own and have an inalienable right to control all of our personal information. But let’s face the harsh reality. This view is naïve and impractical in today’s society. The forces that collect and share personal information are so pervasive and growing so quickly that we can’t even make comprehensive lists of the information-gathering systems, much less define what kinds of collecting and sharing is acceptable.

Think about the three examples I’ve just discussed. We all routinely sign the waivers that allow providers and insurance companies to share our medical records. Have you read one of these waivers? Usually they allow companies to use all forms of records for any purpose for an indefinite period. Just try objecting to the wording on the waiver, especially if you are in the emergency room … And, honestly, the providers and the insurance companies have a right to own the information because they have committed their resources and exposed themselves to liability on your behalf.

Similarly, the retailer has a right to know who you are and what you bought if you expect any form of credit or delivery relationship with it. If you don’t want personalized service, then engage only in anonymous cash transactions at traditional brick-and-mortar stores.

And finally, if you use airports, teller machines, or roads, you implicitly agree to accept the surveillance compromise. Any images collected belong to the government or the bank, at least as far as current law is concerned. An odd corollary of being filmed in a public place is the experience we all have had of walking through a “scene” an amateur photographer is filming. Because a third party has innocently captured our image, do we have any rights of ownership to that image?

What is Likely to Happen?

In my opinion, there are two major ways in which privacy laws and practices will be developed. Either our lawmakers will lead the way with such innovative and insightful legislation as COPPA, or the marketplace and media will force organizations to adapt to our citizens’ perceived privacy concerns. Neither method is likely to produce a perfect solution; however, because we are in an election year, the marketplace and the media seem the likely leaders for the next round of innovation, and the Clinton-Gore privacy initiatives are already engendering a partisan standoff.

David Brin presents a pragmatic and compelling perspective on the threats to privacy and the impact of new technologies in The Transparent Society: Will Technology Force Us to Choose Between Privacy and Freedom? (Perseus Books, 1999). Brin argues that we can strike an effective compromise between freedom and privacy by “watching the watchers.” In other words, we can insist on very visible notifications of information gathering wherever it occurs; honesty and ethical consistency in following the stated policies; and, significantly, on being notified whenever anyone uses our personal information.

The Impact on Webhouse Architecture

The privacy movement is a potent force that may develop quickly. As Webhouse designers, management may suddenly ask us to respond to an array of privacy concerns. How would privacy issues affect our Webhouse? Here are my predictions:

• We will need to consolidate and centralize all the personal information scattered across our organization into a single database. There should only be one consistent, clean set of data about individuals, and we should remove all data that no one is using for any identified purpose from all databases.

• We’ll need to define, enforce, and audit security roles and policies surrounding this centralized personal information database.

• We’ll need to physically isolate the server containing the centralized personal information database on its own segment of the local area network behind a packet filtering gateway that only accepts packets from trusted application servers on the outside.

• We’ll need strong forms of physical and logical security for the backup and recovery of the centralized personal information server.

• We’ll need to define at least two levels of security sensitivity to implement a new privacy standard in our organization. We’ll assign general demographic information a lower level of security; we’ll assign names, account numbers, and selected financial and health-related information a higher level of security.

• An audit database that tracks every use of the personal information must accompany the main database. This audit database must notify every individual of all uses of their personal information, including who requested the information and the type of application. The audit database may have different access requirements from those of the main database. If the audit database is used in a “batch” mode, it pumps out usage reports that it emails (or postally mails) to the individual whose information is being used. If the audit database can be queried online, then it is inherently less secure than the main database and probably needs to sit on a different, more public server. It is important that the audit database contain as little compromising content as possible, and focus on simply disclosing the final uses of information.

• We must provide an interface that authenticates the individual requestor and then provides a copy of all of their personal information stored on the database. A second interface must allow the individual to challenge, comment on, or correct the information.

• We must create a mechanism for effectively expunging information that we deem incorrect, legally inadmissible, or outdated.

Although the data warehouse (and Webhouse) community hasn’t traditionally led the way in advocating social change, I think that it may be a canny look into the future if we each consider whether we could implement any of the changes I’ve mentioned into our organizations. Consider it a “reasonable future scenario” that merits a little advanced planning. If you are more daring, and if you think the privacy debate will end up as the kind of compromise Brin describes in his book, then have a talk with your CIO and your marketing management about some of these ideas.

I am indebted to my son, Brian, for urging the old man to think about some of these issues and for recommending David Brin’s book.