anarchists, established what would be named the General Intelligence Division within the Bureau and put in charge a young Justice Department lawyer named John Edgar Hoover. Hoover set about creating and cross-indexing intelligence files on anarchist leaders and groups. Within two years his directory included 450,000 individuals, organizations, and publications.

During the winter of 1919-20, Hoover, along with the Bureau of Immigration’s head, Anthony Caminetti, directed a series of raids that netted thousands of alleged Bolsheviks and anarchists. What came to be known as the Palmer Raids were initially applauded in the press. The Washington Post declared, “There is no time for hair-splitting over infringements of liberty.” When the details emerged, though, they left a bad taste. Its own manpower limited, the Bureau had relied on local police and vigilante groups. Many of those apprehended were poor and illiterate, their connections to anarchism tenuous. Citizens had been hauled in along with foreigners. The dragnet’s failure to uncover any bombs or stockpiles of weapons gave the lie to the warnings of imminent revolution; warrantless arrests, forced confessions, suspension of legal counsel, and prisoners marched through the streets in shackles awakened fears of a European-style secret police.

The corruption-ridden harding years brought further scandal, and by 1924, Coolidge’s Attorney General, Harlan Fiske Stone, was determined to end the abuses. “... A secret police may become a menace to free governments and free institutions,” he stated.

He wanted a bureau that was “not concerned with political or other opinions of individuals....Only with their conduct and then only with such conduct as is forbidden by the laws of the United States.” Stone directed the 29-year-old Hoover to clean house. In spite of his involvement in the Palmer Raid abuses, or perhaps because of it, Hoover was determined to make the agency incorruptible, tightly controlled, and dedicated to law enforcement rather than spying.

“The Bureau must be divorced from politics and not be a catch-all for political hacks,” he insisted. He culled the bad apples, recruited men of high character trained in accounting or the law, and fired the organization with his enthusiasm. For the next 10 years Hoover kept out of the limelight. He cut back the number of agents and actually returned budgeted funds that he hadn’t spent. He set up a central fingerprint registry and a system to track crime statistics. Later, he would establish the National Police Academy to train law enforcement officers.

Born in Washington, D.C., in 1895, Hoover had grown up in a world of civil servants. He had begun his adult life as a junior messenger at the Library of Congress, eventually working with its card catalogue, all while attending law school. Some biographers suggest that this led to his lifelong obsession with accumulating information and keeping lists: the Gustodial Detention Index, the Rabble-Rouser Index, the Key Activist Index.

Although he came to be a master manipulator of the political system, Hoover scorned politics; he belonged to no party and never voted. He consistently described his enemies in moral terms—outlaws were

“rats crawling from their hide-outs to gnaw at the vitals of our civilization” —and came to believe that perversity, not social conditions, explained wrongdoing. Communism was not a flawed economic and political system, but “the latest form of the eternal rebellion against authority.”

Hoover’s youth and energy helped him make the agency a model of efficiency, and his genuine skills as a leader kept him in the job for 48 years—too long, as it turned out.

Harlan Stone had tried to draw a clear line between the two purposes of policing, law enforcement and intelligence gathering, but the blurring of that distinction would trouble the Bureau for years to come. Law enforcement brings to justice those who break statutes, from Mafia chieftains to venal politicians. In our society the process is open to public view, governed by rules of evidence, overseen by courts, and designed to affect only those reasonably suspected of having committed offenses. By contrast, intelligence gathering, aimed at preventing future threats, is not necessarily tied to specific crimes; its targets can be general: all anarchists, aliens, Communists, Middle Eastern men. The rules of evidence don’t apply, and the methods are necessarily secret, sometimes sinister.

During its first 10 years under Hoover’s direction, the Bureau remained a minor player on the national stage. Then, two events in the mid-thirties transformed its mission in both law enforcement and intelligence.

On the night of July 21, 1934, a group of FBI agents stood sweating on a street on Chicago’s North Side as a man, accompanied by two women, emerged from the Biograph movie theater. At a signal, the agents closed in. When the man began to run, a fusillade dropped him dead on the hot pavement. The agents rushed to phone word of the ambush to their director, who was waiting anxiously in Washington. In minutes, the news spread throughout the country. FBI agents had killed John Dillinger, the man newspapers had called the archcriminal of the age. Decades later, Hoover described the moment as his greatest thrill.

There was a reason the incident shone so brightly in his memory. The “crime wave” of the mid-thirties, really a handful of cases involving hit-and-run kidnappers and bank robbers, shaped the modern FBI. Before 1934, the Bureau was one of a number of federal investigative units. The nation had comparably few federal criminal laws; agents spent much of their time looking into antitrust violations and police-corruption cases. They had no power to make arrests; nor were they authorized to carry guns.

As far as the sketchy statistics of the period indicate, the nation’s crime rate did not accelerate after World War I. The problem was manufactured by a sensationalizing press, fueled by the dislocation of the Depression, and exploited by New Dealers looking to demonstrate the potency of the federal government. The drumbeat for action began in 1932

with the abduction and murder of Charles Lindbergh’s infant son. It picked up a year later when four Bureau men transporting a prisoner ran into an ambush in Kansas City. One agent, the prisoner, and three others died in the “massacre,” and the public started to sense that the government was losing its grip on order in the country. Always quick with hyperbole, Hoover spoke of “a challenge to law and order and civilization itself.”

Until Dillinger’s demise, Homer Cummings, Roosevelt’s energetic Attorney General, had led the anti-crime effort. He had pushed a package of legislation through Congress making bank robbery, flight to avoid prosecution, interstate racketeering, and other misdeeds federal crimes. The anti-crime bills were “one of the most important, if least recognized, New Deal reforms,” notes the historian Sanford Ungar. The legislation gave the FBI real police powers, setting it up as the most visible law-enforcement arm of the Executive Branch.

When kidnappers grabbed Charles Urschel, an Oklahoma oil magnate, in 1933, the Bureau’s meticulous methods had paid off in the arrest of George ("Machine Gun") Kelly, one of the kidnappers. Surrendering, Kelly was reported, probably spuriously, to have pleaded, “Don’t shoot, G-men.” This shorthand for “government men” became the headline-friendly moniker for the members of a federal agency that was about to gain national prominence.

With fewer than 400 agents in 1934, the FBI could hardly police the whole country. That wasn’t the point. What FDR wanted was theater, and Hoover obliged. He selected cases that guaranteed publicity. The slaying of Dillinger and the hunt for various “public enemies”—Pretty Boy Floyd, Baby Face Nelson, the Barker gang—made the Bureau famous (after several name changes, it finally became the Federal Bureau of Investigation in 1935). As movies and magazines trumpeted the G-man myth, newspapers proclaimed Hoover “Public Hero Number One.”

Hoover continued to believe that the Bureau should be a leader in law enforcement, but not a comprehensive federal police force. The agency stayed out of the unpopular business of Prohibition, and Hoover saw no role for it in addressing strictly local crimes like gambling and labor racketeering. Rather, the Bureau should remain an elite agency guiding the nation’s police in modern crime-fighting methods. “Before science all things must fall,” was one of Hoover’s favorite mottoes.

Some New Deal aides had been pushing for the federal government to take a preventive or social response to wrongdoing—attacking problems like slums, poverty, and unemployment that were seen as the roots of crime. The G-man clamor drowned out their voices, touting instead a strategy of prosecuting criminals. The policing philosophy defined the FBI; and the Bureau actively promoted that philosophy.

Two years after the Dillinger coup, as the “crime wave” that had established the Bureau was sputtering out, an equally momentous development in FBI history occurred, far from the glare of publicity. The date was August 24,1936. Concerned about the war clouds gathering in Europe and Asia, President Roosevelt wanted an assessment of the threats from both Communists and fascists in the

United States. Organizations like the right-wing American Liberty League, backed by powerful business executives, were in fact mulling over plots to depose the president.

Because his appropriations did not cover such investigations, Hoover suggested that his authority should rely on a 1916 law authorizing the State Department to approve intelligence operations against agents of foreign powers. The next day, Roosevelt had Secretary of State Cordell Hull give the order—but Hull never put the mandate in writing. In spite of the fact that the law was intended to authorize surveillance only of foreign spies, the Bureau immediately turned its attention to purely domestic groups. The verbal directive, Hoover’s biographer Richard Gid Powers points out, “provided Hoover with his basic authority for nearly forty years of domestic intelligence operations.”

Both Hoover and FDR knew that domestic spying was a political minefield, yet the director was eager to serve the president, and the mission fitted his own inclination to get the goods on suspected radicals. Abandoning his innate caution, he steered the Bureau onto legal thin ice.

As war threatened to engulf Europe in June 1939, Roosevelt issued an official but secret edict directing the FBI to investigate domestic espionage and sabotage. Hoover re-established the General Intelligence Division, the spy unit that had been eliminated in the 1924 reforms; he had conveniently saved its files. The Bureau’s intelligence activities were carried out under the guise of national security, but their targets were often political: Roosevelt critics like Senator Burton Wheeler; isolationists like Charles Lindbergh; labor leaders like John L. Lewis.

During the war years, the Bureau handled counterintelligence efforts throughout the Western Hemisphere and spoiled at least one German sabotage attempt. Hoover opposed as both pointless and illegal the policy of transporting everyone of Japanese descent on the West Coast, including U.S. citizens, to camps far inland. But he established a system of secret files that were not included in the Bureau’s careful indexing system, hiding all its illegal intelligence activity from scrutiny. Congress had outlawed wiretapping in 1934, but Roosevelt decided the ban didn’t apply to national security threats. The FBI arranged for the delay and examination of Axis and Soviet cables, the opening of mail to Axis diplomats, and break-ins for espionage purposes. These activities were illegal, but many continued after the war. When Attorney General Francis Biddle found out about the Bureau’s Custodial Detention Index, listing suspects who were to be rounded up in an emergency, he labeled it “dangerous, illegal” and ordered its end. Hoover simply changed its name to the Security Index.

Had Americans known of the FBI’s clandestine ways during this era, few would have questioned them. Such abuses of power paled alongside the external threats the nation faced, and the FBI’s civil liberties record was still better than that of most local police agencies. Later, the danger of giving the Bureau ill-defined spy duties became clearer. The FBI emerged from the war with some 4,000 agents and a formidable intelligence capability. Hoover maneuvered to maintain an FBI monopoly over domestic

intelligence, and Congress restricted the new Central Intelligence Agency to overseas duty. Tension with a resurgent Soviet empire convinced the Director that another war was imminent and that the Bureau needed to be ready.

The Smith Act, passed in 1940, had made it a crime to advocate, or to belong to a group that advocated, the overthrow of the government by force. It provided a potent weapon against radicals during the Cold War. The Bureau targeted and won convictions of top leaders of the American Communist movement in 1948 and 1949, and additional party members went to trial over the next five years. The FBI also solved what Hoover at the time called the “crime of the century,” the theft of America’s atomic bomb secrets. With the help of meticulous code-breaking, agents were able to alert British authorities to the spy Klaus Fuchs and to apprehend his U.S. contacts and co-conspirators, including Julius and Ethel Rosenberg.

After the war, President Truman renewed his predecessor’s order permitting wiretaps for intelligence purposes. Hoover latched on to accusations against purported Communists in government and assigned 300 agents to investigate the former State Department aide Alger Hiss, one of the highest government officials accused of Communist-party involvement. Hiss was convicted of perjury in January of 1950 and jailed. Convinced that the Truman administration was not doing enough to counter the Red threat, Hoover fed FBI data to the House Un-American Activities Committee, letting representatives attack with innuendo citizens whom the Bureau lacked the evidence to prosecute. In addition, Joseph McCarthy’s Senate subcommittee on investigations employed ex-FBI agents as sleuths and made liberal use of information from Bureau files (although when the senator began to make accusations against the Eisenhower administration itself, Hoover backed off).

The Bureau set up a Responsibilities Program under which numerous school teachers, college professors, and state government workers lost their jobs. Suspicions multiplied. Writers, reporters, and entertainers came under FBI scrutiny. The Bureau kept files on Leonard Bernstein and Pete Seeger, on Charlie Chaplin and Ernest Hemingway. It had drifted far from its law enforcement duties. It was about to stray even farther.

In the early 1960s, as the nation experienced a serious increase in crime, the Kennedy administration, searching for solutions, finally began to look at possible social causes of criminal behavior. But, with Hoover still blaming crime on the moral failings of wrongdoers, the FBI continued to lobby for heftier Bureau appropriations with which to lock up more criminals. The agency had always been the nemesis of bank robbers, kidnappers, and car thieves, whose cases could be solved with relative ease and offered high conviction rates. For decades, however, it had failed to pursue organized crime. Hoover had labeled the notion of a national crime syndicate “baloney.”

During the 1930s, urban gangs had used the clout and cash they gained during Prohibition to move into extortion, gambling, narcotics, and other rackets. Mysteries, some

salacious, have been spun about Hoover’s reluctance to go after gangsters, but several straightforward explanations suggest themselves. Mob cases required tedious legwork and involved the danger that gangsters might corrupt agents. Organized crime’s intractability warned Hoover away from a potential quagmire. During the postwar period, an all-out attack on gangsters would have taken Bureau resources away from the anti-Red crusade. As late as 1959, the FBI’s New York office had 400 agents chasing “subversives,” but only four were investigating organized crime.

The discovery of a “crime convention” in the upstate New York hamlet of Apalachin in 1957 by two state troopers and two U.S. Treasury agents embarrassed the Bureau, which had to admit it knew nothing of the conclave. The headlines sparked public concern about the extent of organized crime, forcing Hoover to begin moving against the mob. Under his Top Hoodlums program, wiretaps and bugs proved effective in garnering intelligence about gangsters, but because the methods were illegal, the information could not be used as evidence in court.

In 1968, Congress finally gave the FBI its long-sought legal authorization to carry out court-approved eavesdropping. The Racketeer Influenced and Corrupt Organizations (RICO) provision of the Organized Crime Control Act of 1970 allowed entire criminal enterprises to be indicted, and the gangster tide began to recede. The FBI gathered evidence that led to 2,500 convictions between 1981 and 1983, successfully hauled eight members of the Mafia “Commission” into court in 1986, and continued to hammer mob bosses through the 1990s.

Civil rights was another area where the FBI initially feared to tread. The Bureau was criticized for standing by in the face of provocations like the violence against the Freedom Riders in 1961 and Alabama’s official defiance of the integration of its state university in 1963. But, under pressure from President Johnson, the Bureau finally came to play a significant role in suppressing the violent reaction to the civil rights movement in the South.

Throughout the 50s and 60s, as Hoover maintained the FBI’s emphasis on intelligence activities, his vigilance regarding domestic Communism approached paranoia. The fact was that the Communist threat at home had largely evaporated. The Smith Act trials had decapitated the party in the United States, and revelations of Stalin’s crimes, combined with the Soviet suppression of the 1956 Hungarian revolt, had seriously sapped party morale. By the end of 1957, the party claimed fewer than 4,000 members, many of them FBI informants. Nevertheless, hoping to obliterate the menace once and for all, the Bureau began the most serious intrusion ever of police power into the American political process. It was to be one of the FBI’s deepest secrets, with all actions personally approved by the Director. It became known as it COINTELPRO, for “counter-intelligence program.”

Not content with investigation, Hoover now ordered his men to infiltrate and disrupt the activities of

suspected organizations. The point was not to prosecute criminals but to hamstring political activities. Agents made anonymous phone calls and wrote letters denouncing targets in sexual terms. They contacted neighbors, employers, and friends to spread the word that a person was disloyal. They encouraged informants to sow discord, which sometimes led to violence. They arranged for targeted tax audits, and put “snitch jackets” on people, planting false rumors that they were cooperating with authorities.

The tactics were so effective that they spawned, over the next 15 years, additional COINTELPROs, targeting the Socialist Workers party, white hate groups, black militants, the “New Left,” and others. The Communist-party COINTELPRO, though unauthorized, had tenuous justification in the Communist Control Act of 1954, which had outlawed the party. But,with each subsequent program, the agency moved farther away from any mooring to legality.

The FBI’s most notorious abuses were directed at Martin Luther King, Jr., who became the focus of the Bureau’s interest as early as 1958. He thrice transgressed: He was black, Hoover thought he had Communist ties, and he had the temerity to criticize the Bureau. In spite of the fact that its own studies had shown few connections between Communists and civil rights groups, the FBI tapped phones and hid microphones in King’s hotel rooms. It distributed tapes and reports from these sources as part of a vigorous campaign to demoralize the civil rights leader and undermine his reputation. Hoover held a press conference on November 18,1964 in which he called King “one of the lowest characters in the country.” But, by the mid-1960s, these red-baiting techniques had begun to lose their potency; Hoover was falling out of sync with the temper of the times.

Once the FBI entered the business of combating political opinions, it became a criminal conspiracy itself. In March 1971, a group called the Citizens’ Commission to Investigate the FBI burglarized a Bureau office in Media, Pennsylvania, and made off with heaps of secret files. The burglary prompted Hoover to shut down his most egregious domestic spy operations. While the security breach barely made news, the designation COINTELPRO on one of the stolen documents, which the group sent to newspaper and television reporters, proved critical. No one at the time knew what the term meant, but the NBC News reporter Carl Stern made Freedom of Information Act filings that gradually helped reveal a long history of illegal activity. The Media break-in proved to be, in the words of one Bureau official, the “turning point in the FBI image,” forever tarnishing the G-man myth. By the time all the facts emerged, Hoover was dead and Watergate had darkened the national mood.

In 1976, Attorney General Edward Levi moved to re-establish Justice Department control over the FBI. His guidelines, echoing the sentiments of Attorney General Stone 52 years earlier, set limits on operations not specifically directed at criminal activity. They also required the Bureau to clear its investigations in advance with the Justice Department. Congress had already

limited the director’s term to 10 years, with Senate confirmation of his appointment.

Seven years later, President Reagan’s attorney general William French Smith rescinded Levi’s orders, once again letting the Bureau dig into the affairs of persons or groups that “advocate criminal activity.” The FBI had only to notify the Justice Department, not seek permission from it, when initiating a domestic-security probe.

One effect of Watergate was the passage of the Foreign Intelligence Surveillance Act of 1978, an effort to bring legal order to the intelligence realm. The FISA set up a secret court to handle requests for investigations of alleged agents of foreign powers. Warrants for wiretaps for intelligence gathering, rather than law enforcement needed no probable cause. The law recognized that the Bureau would have to continue to pursue intelligence in national security cases, but attempted to contain its methods by limiting them to use against foreign agents.

The scandals of the 1970s and the resulting legislative restrictions temporarily damped the Bureau’s domestic intelligence activity and brought law enforcement back to the fore. Organized crime, white-collar crime, and tracking down foreign spies became top priorities. The Bureau introduced new methods like psychological profiling and, during the 1980s, DNA analysis. It also opened its ranks to minorities and women.

In the 1978-80 ABSCAM investigations, agents impersonating the representatives of a fictitious Arab sheik caught a U.S. senator, six representatives, and several others taking bribes. Operations against corrupt local government officials followed. Political malfeasance had long been a Bureau responsibility but one that it had approached gingerly.

By 1982, terrorism had emerged as a prominent focus of Bureau attention, and with it came a renewal of intelligence operations. The FBI prevented a reported 131 terrorist attacks between 1981 and 2000. It foiled a 1999 plot to blow up a Sacramento gas tank that could have killed 12,000 people. It helped to stymie a wave of planned millennium terror attacks on the West Coast. It solved the 1993 bombing of the World Trade Center in a matter of weeks and pursued the case to successful convictions, heading off additional attacks in the process. The perpetrator of the Oklahoma City bombing two years later was also quickly apprehended, but the incident further emphasized the need for effective preventive intelligence. In the meantime the CIA was chagrined by the discovery of the double agent Aldrich Ames, and in 1994, President Clinton gave the FBI more counter-intelligence responsibility, making it the lead agency for combating terrorism and crime worldwide. The Bureau began to expand its presence overseas; by last year, it operated 44 offices in foreign countries.

In 1994, the Bureau’s director, Louis Freeh, warned that the nation needed to exercise “constant vigilance” against terrorism. Two years later ,Congress chimed in by authorizing the Attorney General to investigate U.S. citizens suspected of supporting terrorist groups. It also lifted restrictions that had forbidden the FBI from spying on a citizen solely because he or she had taken actions protected by the First Amendment. The

Bureau began for the first time to feed intelligence data into the National Crime Information Center system, which had been limited to criminal records.

In 1996, the FBI set up a counter-terrorism center to coordinate the efforts of analysts from 16 agencies and local police forces, and the number of agents assigned to intelligence grew from 224 in 1992 to 1,025 by the end of the decade. In 2000, counter-intelligence and the fight against terrorism were designated the Bureau’s top priorities.

The stepped-up intelligence effort required a difficult adjustment in FBI culture. Law enforcement had always been the path by which agents advanced. Solving cases led to promotion, while poring over scholarly journals and foreign newspapers, the legwork of intelligence gathering, was nobody’s dream of glory. Data analysis was a long-standing FBI weakness. Moreover, now the Bureau had to work closely with other agencies, something it had traditionally resisted. ”... No one institution is strong enough to tackle the challenge of terrorism alone,” Director Mueller emphasized. “Law enforcement, quite simply, is only as good as its relationships.”

If inappropriate domestic spying had earlier been the principal complaint against the Bureau, mismanagement became the dominant charge during the 1990s. In 1993, the FBI failed to resolve a confrontation with religious fanatics in Waco, Texas, and the standoff ended in the incineration of 76 people, 25 of them children. The “shoot on sight” order issued during the siege of a militant’s home in Ruby Ridge, Idaho in 1992 had also cost innocent lives. However, the Bureau did learn from these incidents, and it handled a 1996 Montana showdown involving the group that called itself the Freemen with patient and ultimately successful restraint.

During the same period, even the Bureau’s vaunted crime lab suffered from mismanagement. The spy Robert Hanssen was revealed to have operated undetected within the Bureau’s own counter-intelligence section for 21 years. And the handling of the case of the atomic scientist and one-time suspected spy Wen Ho Lee “embarrassed this entire nation,” a district court judge said.

Critics noted that part of the problem lay in the ever-widening scope of FBI responsibility. Some enforcement efforts are by nature federal; they include corruption among government officials; civil rights violations; and national or international crime conspiracies. But Congress may have turned to the FBI for solutions too many times. It made car-jacking a federal crime. telemarketing fraud, child pornography, and money laundering were added to its list, as was economic espionage. Parents who neglected to pay child support became FBI targets. Drug trafficking grew to account for nearly 26 percent of the Bureau’s convictions by 1998 and as many as 80 percent in some districts.

Last fall’s terrorist attacks have prompted Director Mueller to take another look at some of these duties. Agents have been shifted from drug enforcement, white-collar fraud, and violent crime to the anti-terror beat. Even bank robbery,

long a source of Bureau glory, may in some cases be left to local police.

The crisis has given both focus and urgency to Mueller’s mission to overhaul the agency he heads. Some reforms address long-standing weaknesses: He has proposed new sections devoted to cybercrime and to the Bureau’s own internal security. An Office of Law Enforcement Coordination will help the FBI better mesh with local police. Bringing the Bureau’s computers up to speed has become a priority.

In May, Mueller announced a further reorganization that he asserted will result in a “redesigned and refocused FBI.” A Justice Department official described the shifts as “turning the ship 180 degrees from prosecution of crimes as our main focus to the prevention of terrorist acts.” At the same time, Attorney General John Ashcroft announced that he was loosening restrictions on FBI investigations, thereby broadening the Bureau’s ability to conduct domestic intelligence surveillance unrelated to criminal activity.

Preventing terror, the Bureau’s number one priority, will now be the focus of 2,600 agents, more than double the number who worked in the area before September 11. Hundreds of newly hired analysts, linguists, and computer experts will labor to make sense of the data compiled by agents. CIA personnel will work directly with the FBI in the Bureau’s new Office of Intelligence.

Some critics have asked whether the shakeup goes far enough. Others question the wisdom of encouraging the Bureau to increase its domestic spying given that its intelligence failures before the September attacks were related more to bureaucratic bungling than to inadequate information-gathering authority.

In the end, the questions facing Mueller, and the nation itself, are those that have hung over the FBI since its earliest days: What kind of federal police force do we want? What portion of our freedom and privacy will we relinquish in exchange for security? Can we trust our government with the secret and sometimes dirty tools of intelligence? The Bureau’s history serves as a cautionary backdrop for the changes that lie ahead.