My interpretation of this article and the "Top Secret America" articles also published by the Washington Post. Is that the DNI (Director of National Intelligence) is overwhelmed by the amount of data and articles published by the various Intelligence agencies that it is nearly impossible to "connect the dots" and finding terrorists before they attack. Although that may be true there have been no successful attacks on the U.S.A since the 9/11 attacks so they must be able to glean something of value from what data they have.
Considering the above listed problem of data overwhelming the agency, I have developed an idea for an organizational structure that should be able to help in putting the information at hand to good use in defense of the U.S.A. Of course I have no interest and prefer "turf wars" not happen and am going to ignore that for the time being, but will address it at the end in a more complete manner.
Basically the concept of the structure is to be a service organization, not a computer service, but a people service. I know coming from a software developer a "manual" versus and automated service may sound like I am nuts, but being in the industry for years and working towards the Holy Grail of Software Devopment by developing my own HAL 9000 computer system is not a realistic goal. I am sure the geeks over at NSA are working on the prototype for HAL 7000 right now and think I am selling them short. No I think improvements in HAL like systems will certainly enhance the Information Worker but never take his or her place in this world.
The organization will have to not be like any existing agency; it's people will need to have an outward face welcoming and helpful to the Intelligence community seeking their help. This agencies workers need to feel and be proud of the fact that they are helping defend this wonderful country.
The sole purpose of the workers is to help the Intelligence Agents that are seeking information about subjects or that are trying to generate leads on possible threats to gather information from the vast archive of articles maintained by the agency. The workers are to be Librarians who are also up to date on the jargon of the industry and are already well read on the subjects in question and can talk intelligently to the agents seeking their help. There should be Librarians that are available to the Librarians who are more subject matter experts, so when a Librarian is helping an agent, they may seek further help with the subject matter experts to help glean more information.
Now how does this help deal with the fact the agency is overwhelmed with data. First let me say there needs to be some software written to support the Librarians. The software support the Librarians in a manner like the Open Directory Project (DMOZ) where human interpreters are trying to categorize manually the web, a human run Google if you will. In this agency those human interpreters read all incoming articles and categorize and tag the articles with search terms that would help future searches in regards to those articles. Typically I see the subject matter experts reading all articles in a certain main subject and doing the categorization and tagging. This would mean there is a vast amount of knowledge already in their heads when they are approached by the Librarians asking for help.
In regards to categorization and tagging, I see this as likely a fluid environment, as current events change priorities the tags and categories may change. New tags will come along and as old articles are brought up in current searches they should be re-tagged by the subject matter experts with new tags as necessary. I see this being done not on all old articles but only on old ones that are brought up by new searches. This way the agency does not need to re-churn the entire archive every time new tags are added to the system.
Now about security, which would be of great concern for the folks in the Intelligence Agencies I have some possible security measures that can be developed into it. First the agency will have a search portal like Google usually run by the Librarians but can also be run by outside agencies. Assuming the structure is that an article is classified by a security level such as an integer, and an individual can only see articles at his or her security level or below this can be developed into the system. First the various Librarians should have sufficiently high security levels to view and categorize most articles in the system and a few Librarians that can work on the few that are of high level security. When a search is performed the articles that the Librarian and/or the agent can view are displayed. But also any articles that they cannot view because of security level will be listed as a number such as "There are 5 articles that may be of help, but cannot be viewed because of security concerns." Then a Librarian of a higher security level can review the articles and determine if they are of value and if they are of value, a determination by an appropriate authority can be made to decide if the agents can view the articles or not.
Dealing with "turf war" problems. Most bureaucrats like to strongly defend their "turf" and are not likely to give another bureaucrat from another "turf" anything useful. To deal with this I have some ideas, but not being a bureaucrat I do not know if it will fly or not. Pay incentives, all of our Librarians and subject matter experts will be given a base pay. But at each juncture with an agent from one of the other agencies, if the other agent feels the information was helpful to them they would indicate so and each time they get this type of acknowlegement they get a micro bonus (not sure what the actual amount will be). Make the micro bonuses be such that in any given pay period the worker can get maybe up to 50% more money then their base pay if they are being helpful.
Anyway please comment.
Bodger
