Should people have the right to discriminate by race, sex, religion and other attributes? In a free society, I say yes. Let's look at it. When I was selecting a marriage partner, I systematically discriminated against white women, Asian women and women of other ethnicities that I found less preferable. The Nation of Islam discriminates against white members. The Aryan Brotherhood discriminates against having black members. The Ku Klux Klan discriminates against having Catholic and Jewish members. The NFL discriminates against hiring female quarterbacks. The NAACP National Board of Directors, at least according to the photo on their Web page, has no white members.
You say, Williams, that's different. It's not like public transportation, restaurants and hotel service in which Title II of the 1964 Civil Rights Act "prohibits discrimination because of race, color, religion, or national origin in certain places of public accommodation, such as hotels, restaurants, and places of entertainment." While there are many places that serve the public, it doesn't change the fact that they are privately owned, and who is admitted, under what conditions, should be up to the owner.
If places of public accommodation were free to racially discriminate, how much racial discrimination would there be? In answering that question, we should acknowledge that just because a person is free to do something, it doesn't follow that he will find it in his interest to do so. An interesting example is found in an article by Dr. Jennifer Roback titled "The Political Economy of Segregation: The Case of Segregated Streetcars," in Journal of Economic History (1986). During the late 1800s, private streetcar companies in Augusta, Houston, Jacksonville, Mobile, Montgomery and Memphis were not segregated, but by the early 1900s, they were. Why? City ordinances forced them to segregate black and white passengers. Numerous Jim Crow laws ruled the day throughout the South mandating segregation in public accommodations.
When one sees a law on the books, he should suspect that the law is there because not everyone would voluntarily comply with the law's specifications. Extra-legal measures, that included violence, backed up Jim Crow laws. When white solidarity is confronted by the specter of higher profits by serving blacks, it's likely that profits will win. Thus, Title II of the 1964 Civil Rights represented government countering government-backed Jim Crow laws.
One does not have to be a racist to recognize that the federal government has no constitutional authority to prohibit racial or any other kind of discrimination by private parties. Moreover, the true test of one's commitment to freedom of association doesn't come when he permits people to associate in ways he deems appropriate. It comes when he permits people to voluntarily associate in ways he deems offensive.