Messages like that are not the way Richardson built a Spartanburg, S.C., hamburger stand into one of the nation’s largest food-service companies. But a widening circle of allegations that Denny’s has discriminated against black customers has called into question the future of Richardson’s Flagstar Cos. Richardson, whose reputation for fancy footwork dates to when he caught a Johnny Unitas touchdown pass in the 1959 NFL championship, will need all the moves he can muster to gussy up the image of his $3.7 billion empire.

Flagstar has been under the gun since late 1991, when an employee at a Denny’s in San Jose, Calif., allegedly told a group of black high schoolers that “company policy” required them to pay for their late-night snacks in advance. Then, last May, just as Flagstar seemed to have put its image problems behind it by signing a nondiscrimination agreement with the U.S. Department of justice, six black Secret Service officers charged that they had been ignored at a Maryland Denny’s while white officers seated nearby were served promptly. Plaintiffs’ lawyers have been enjoying a field day ever since, collecting affidavits about 50 alleged incidents of discrimination in 13 states.

While mounting a public-relations defense, Flagstar contends that any discrimination problems are isolated. “It does not make sense for this company not to want to serve everybody that comes into the restaurant,” Richardson insists. During an interview with NEWSWEEK in his Spartanburg office, Richardson whipped out pen and paper to sketch the layout of the Maryland Denny’s and assert that even a large all-white group would have been subjected to the same bad service as the black Secret Service officers. “We had one cook, and either two or three servers to serve the entire restaurant,” he says. “If they say they were discriminated against, I apologize. But in my opinion, there was not an intent to not serve black people.”

While the lawsuits may not be heard for years, Denny’s already soft business is suffering the consequences. At his franchise in suburban Atlanta, general manager Steve Tedder says sales are down about 10 percent, owing largely to the discrimination claims. Sales at Denny’s outlets in northern California are off as well. Flagstar’s consumer research shows that the allegations are well known to customers. The company insists that the damage has not spilled over to its other chains, which include Quincy’s Family Steakhouses and Hardee’s restaurants. But a third of Flagstar’s sales come from its Canteen division, which operates concessions at parks, stadiums and other publicly owned sites; if the company’s problems lead local politicians to make an issue of Canteen’s franchises, Flagstar could be hurt badly. Investors are expecting the worst: Flagstar’s shares are trading near the lowest price since Richardson took the company public in 1989.

Richardson’s other interests have been swept into the controversy. The 56-year-old is leading an effort to bring an NFL franchise to Charlotte, N.C. To win favor from a league highly sensitive to racial issues, Richardson two weeks ago signed agreements with the National Association for the Advancement of Colored People, pledging to increase jobs and management opportunities for minorities at both Flagstar and the football team. The agreements have raised a furor in Baltimore, site of the NAACP’s headquarters–and a competitor of Charlotte’s for the new franchise. But NAACP executive director Benjamin Chavis is unrepentant. “I have observed in Richardson over the last two months a level of sincerity unmatched by a corporate executive,” he says.

Richardson takes the blame for Flag-star’s debacle. The company has come through a decade of nonstop financial machinations: Richardson sold it in 1979, regained it through a buyout in 1989, took it public and then. last year, induced buyout kings Kohlberg Kravis Roberts to invest $300 million for a 47 percent stake. Dealing endlessly with finances, he says, blinded him to other aspects of running a corporation in a culturally diverse America, like the presence of only one black among owners of the 378 U.S. Denny’s franchises. “I should have invested the time earlier, and I regret it,” says Richardson. Unfortunately for Flagstar and Richardson, time may be running out.

Denny’s is in the spotlight. But Flagstar Cos., with $3.7 billion in sales, has built a food-service empire that includes everything from steaks to broiled chicken.

The $1.45 billion food chain operates more than 1.400 outlets that feature moderate prices and table service. Its slogan: “Always Open.”

Flagstar is the hamburger merchant’s largest franchisee, operating 528 of these fast-food outlets–mostly in the Southeast. Sales: $607 million.

The 217-outlet beef chain produced sales of $290 million last year. The price of a New York strip sirloin and baked potato: $9.95.

Its 210 fast-food restaurants, most of them based in southern California, are known for flame-broiled chicken. Sales: $98 million.

Flagstar’s second largest unit, with sales of $1.3 billion, provides food service in places like Yankee Stadium and Yellowstone National Park.