White House Hacks White House Site
Quote:
Enabling historical revisionism
Via Atrios, we see that the White House has edited its website to keep search engines from archiving pages on Iraq.
First, a bit of technical background. Most major websites include a text file named robots.txt that tells search engines which directories not to include in search results. (Here's an example: the Democrats.org robots.txt file lists folders with content ? like images ? that search engines can't index.) By adding a directory to robots.txt, you ensure that nothing in that folder will ever show up in a Google search and ? more important for this discussion ? never be archived by sites like Google.
Sometime between April 2003 and October 2003, someone at the White House added virtually all of the directories with "Iraq" in them to itsrobots.txt file, meaning that search engines would no longer list those pages in results or archive them.
Why would the White House do this? Those pages are still public, and the White House search engine itself does index those pages, so users can still get to them.
It's easy enough to understand the reasoning if you look at past White House actions. Earlier this year, the White House revised pages on its website claiming that "combat" was over in Iraq, changing them to say "major combat."
One of the reasons some alert readers noticed the change ? and were able to prove it ? was that Google had archived the pages before the change occurred. Now that all of the White House pages about Iraq are no longer archived by Google, such historical revisionism will be harder to catch.
Enabling historical revisionism
Via Atrios, we see that the White House has edited its website to keep search engines from archiving pages on Iraq.
First, a bit of technical background. Most major websites include a text file named robots.txt that tells search engines which directories not to include in search results. (Here's an example: the Democrats.org robots.txt file lists folders with content ? like images ? that search engines can't index.) By adding a directory to robots.txt, you ensure that nothing in that folder will ever show up in a Google search and ? more important for this discussion ? never be archived by sites like Google.
Sometime between April 2003 and October 2003, someone at the White House added virtually all of the directories with "Iraq" in them to itsrobots.txt file, meaning that search engines would no longer list those pages in results or archive them.
Why would the White House do this? Those pages are still public, and the White House search engine itself does index those pages, so users can still get to them.
It's easy enough to understand the reasoning if you look at past White House actions. Earlier this year, the White House revised pages on its website claiming that "combat" was over in Iraq, changing them to say "major combat."
One of the reasons some alert readers noticed the change ? and were able to prove it ? was that Google had archived the pages before the change occurred. Now that all of the White House pages about Iraq are no longer archived by Google, such historical revisionism will be harder to catch.
http://www.democrats.org/blog/display/00010130.html
By blocking access to google, the white house can then "revise" what they said in previous versions of the site.
What's next changing the sixteen words in his state of the union to six?
I thought every scrap of paper was supposed to be saved in any administration. Although technically they are still there don't you think this is a little short of "honesty and integrity"?
I think what google does is good for the american public for exactly this reason. To serve as an outside independent source. Now it looks like the white house does not want this to happen...
Does anyone just think there is something wrong with this?
What branch of goverment (besides the white house since we know they don't want to) could pass a law to "full web site access and disclosure"?
Comments
The White House can just revise any Iraq article after the fact without noting the revision. I was telling Fellowship last night....
Oh... my fragile illusions!
Originally posted by groverat
Oh my God! An organization can manipulate its web site to reflect what it wants?
Oh... my fragile illusions!
fantastically glib, groverat!!!
It's denying Google (or any search engine for that matter) the ability to SEARCH the white house site for iraq information. It's denying Google the ability to ARCHIVE White House articles.
"Yeah..um...we never said Iraq was an imminent threat. Check our web site!"
Originally posted by groverat
I didn't know Google had the right to archive everything on the internet.
One would think archiving public white house documents in an easily accessible place would be a good thing. You don't think so? Most of them are still available on the White House site...it just eludes me why the administration thinks restricting availability of information about iraq is a good thing-- beyond obvious political benefits.
Originally posted by groverat
I didn't know Google had the right to archive everything on the internet.
So you have no discomfort with the White House actions?
Having archives is a GOOD thing.
They have to go out of their way to block this info.
Why?
So once again change their version of the truth (at least to google)...
How can anyone find any good in this action?
Originally posted by Rick1138
Everybody gets the government they deserve. Welcome to Soviet America.
Groveratistan.
Click the link to see what the white house does not want 3rd party archivers to see!
http://www.whitehouse.gov/robots.txt
# robots.txt for http://www.whitehouse.gov/
User-agent
Disallow
Disallow
Disallow
Disallow
Disallow
Disallow
Disallow
Disallow
Disallow
Disallow
Disallow
Disallow
Disallow
Disallow
Disallow
Disallow
Disallow
Disallow
Disallow
Disallow
Disallow
Disallow
Disallow
Disallow
Disallow
Disallow
Disallow
Disallow
Disallow
Disallow
Disallow
Disallow
Disallow
Disallow
Disallow
Disallow
Disallow
Disallow
Disallow
Disallow
Disallow
Disallow
Disallow
Disallow
Disallow
Disallow
Disallow
Disallow
Disallow
Disallow
Disallow
Disallow
Disallow
Disallow
Disallow
Disallow
Disallow
Disallow
Disallow
Disallow
Disallow
Disallow
Disallow
Disallow
Disallow
Disallow
Disallow
Disallow
Disallow
Disallow
Disallow
Disallow
Disallow
Disallow
Disallow
Disallow
Disallow
Disallow
Disallow
Disallow
Disallow
Disallow
Disallow
Disallow
Disallow
Disallow
Disallow
Disallow
Disallow
Disallow
Disallow
Disallow
Disallow
Disallow
Disallow
Disallow
Disallow
Disallow
Disallow
Disallow
Disallow
Disallow
Disallow
Disallow
Disallow
It goes on and on and on... VP doesn't like his stuff seen either (Surprise
http://weblog.siliconvalley.com/colu...s/001450.shtml
So you have no discomfort with the White House actions?
I am against locks on information in principle, but I hardly see why this matters. If the White House were keeping what should be public information private, I would have a problem. But these are restricted fluff pieces.
This is absolutely nothing more than Bush-hate finding a new outlet. The fact that they have restricted archiving of their content means nothing.
And the White House isn't actually locking down any content. You can see and archive anything you want.
That way the american people can see me the way i want them to.
It's not like i'm using their tax dollars to block them....
Originally posted by groverat
Harald:
I am against locks on information in principle, but I hardly see why this matters. If the White House were keeping what should be public information private, I would have a problem. But these are restricted fluff pieces.
This is absolutely nothing more than Bush-hate finding a new outlet. The fact that they have restricted archiving of their content means nothing.
And the White House isn't actually locking down any content. You can see and archive anything you want.
And I would argue that your defense is nothing more than Bush-apology finding a new outlet. Hence your refusal to think beyond black and white terms of any amount of availability (good) vs. absolute non-availability (bad). In fact, the White House has made its Iraq documents significantly less available to the public, which means less available to scrutiny by extension. Anyway, you just brush-off the public white house documents as "fluff pieces" when they play a very big role in showing how administration rhetoric lead up to the war in iraq. Of course that would only be regarded as fluff by defenders of the administration (and of the war.)
That way the american people can see me the way i want them to.
The White House is not removing access to content nor is it restricting "the way the American people see" him.
Politicians are politicking. Nothing more.
--
ShawnJ:
Hi, have we met? As a citizen of the US and Texas I have never voted for or even advocated voting for George W Bush.
As far as apologizing for Bush, I'm somehow doubting he had something to do with this. This is a paranoid administration under heavy and constant fire acting like scared politicians. Big shocker.
In fact, the White House has made its Iraq documents significantly less available to the public, which means less available to scrutiny by extension.
How many people go to GeeDub's site to get information? Those people need to seriously improve their methods of information gathering because I can't think of anything more stupid than going to whitehouse.gov to get information on Iraq.
I don't even know why they have a website that discusses these issues, it's propaganda. They aren't letting Google use bots to archive their propaganda! OH NO!
Anyway, you just brush-off the public white house documents as "fluff pieces" when they play a very big role in showing how administration rhetoric lead up to the war in iraq.
Are those documents being removed? Are people not allowed to see them anymore?
------
I would encourage everyone to read the commentary under the linked blog post.
We've got a bunch of
The White House is not removing access to content n
I disagree. the harder you make people find stuff, the more people will give up. Opt out email newsletters for instance. The ones that let you reply with "unsubscribe" are easy to. But the ones that that force you to go to a web site, find the page, then type in some code or your email address are doing so to make it much harder for you to opt out.
I'm still waiting for a post that explains to me why using government employees to go out of their way to block information from one of the largest search companies in the world is a good thing...
10 bucks says ashcroft and co. ties this to terrorists...<cough>
Originally posted by Argentina
So why is this so terrible? Still not really seeing it.
it's not THAT terrible in and of itself, but the implications are kind of disturbing.
Originally posted by Argento
So why is this so terrible? Still not really seeing it.
Because the White House has "revised and extended their remarks" and been caught in the past. It seems like they're trying to make sure they don't get caught again in the future. Why have a public site and yet prevent it from being independently searched and automatically archived, unless you're intent on covering your future tracks in case you need to make any quick changes?
More importantly, they're stonewalling the 9/11 commission, which Bush opposed at the outset, by denying them intelligence documents.
Originally posted by Argento
So why is this so terrible? Still not really seeing it.
If it's not important, then why would they go out of their way to tell someone not to search their site?