MediaWiki talk:Community Portal/Damage Control Central

2009-03-17T01:48:39Z

96.240.208.203: /* Pages "recovered" list */

Additional discussion on this situation can be found on these Allspark.com threads:
* [http://www.allspark.com/forums/index.php?showtopic=56439 "TFWiki Problems, Any idea what's going on?"]
* [http://www.allspark.com/forums/index.php?showtopic=51208&st=400&p=1155749&#entry1155749 "Major Wiki Work, A thread for additions to Transformers wikis."]

==Hey hey, what can I do==
I think at this point we must begin formulating plans based on the assumption that the site is a total loss. This means damage control first, then restoration, then prevention.

Assuming no good news comes through, it seems to me we need to take the following steps. Bein' that I'm not a tech guy, there may be more options than what I'm seeing, but here's what I figure we gotta do:

#take down the bulk of the site's pages so Google's cache will not overwrite them
#figure out a system to make Derik and Abates' caches of Google-archived pages accessible to editors as they come in (Derik mentioned it'll take over a week to get a complete set off Google.)
#if possible, automate the stripping of HTML tags and other non-Wiki goop from those pages before anyone tries to edit them, including Wikia links.
#find a way to divvy up a restoration workload among our 50 or so regular editors.
#find a new server. We would be idiots to remain here.

I still hope most of this will be unnecessary, but at this point, I'm losing faith. -- '''repowers''' [[Special:Contributions/99.140.167.2|99.140.167.2]] 12:08, 16 March 2009 (EDT)
: Spoke with Bookworm several times this morning. He still has a few more computers to search for backups, but, well, at this point I'm going to assume the worst. He is going to try to at least get our user database working. He wasn't aware that was a problem until I let him know. I think Scout is trying to get all the pages downloaded from Google Caching. I've grabbed a few of my own, but she's got, like, thousands of pages so far.
: As for getting a new server... anybody got any ideas? Hell if I know. Frankly, don't listen to me anyway on that. --Walky

::Take everything down immediately. The longer the site stays up as-is, the greater the risk of contaminating Google's March 9th cache and the more linkbacks we send to Wikia. Then we either beg Google for a mass infodump, or.... 50 editors grab 160 pages each. EDIT: or Scout gets them! Thanks, Scout; good to see at least the words "silver lining" in print, even if I don't see the actual thing itself yet.... '''Thylacine 2000'''--[[Special:Contributions/74.73.131.210|74.73.131.210]] 12:17, 16 March 2009 (EDT)
:::If we take everything down, what are we going to edit back together? WHEN are we going to edit everything back together? Taking it down just delays our ability to fix it. --Walky
::::If we've got our copy of the cache, then site shutdown isn't nearly as important. The main concern is that we'd lose the Google cache. Looking like retards... well, we'll just have to take our lumps there. (I guess we could set permissions for content pages to only logged-in users? Dunno.) -- '''repowers''' [[Special:Contributions/99.140.167.2|99.140.167.2]] 12:30, 16 March 2009 (EDT)
:::::Even if it doesn't overwrite the cache, I see no reason to have the site up given the state that it's in. I say take it down until the head-honchos figure out how to clean up this mess. '''(Undecided)''' [[Special:Contributions/67.185.148.107|67.185.148.107]] 16:13, 16 March 2009 (EDT)

==I am a dumb, stupid man==
Just to get that out of the way. Should have told BK to back off on the database upgrade as soon as I saw his original email, and maybe we shouldn't have been here on this server to begin with. It was my call, after all. I'll do my best to make it up to you. Especially TOOG and McFeely, who did so many pages. When I remembered Google caches stuff at like 7 in the morning and started downloading pages whole-hog, theirs was the work I went after first. BEFORE HOT SHOT. That is how bad I feel. Nobody should have to watch through Energon twice. Or, uh, once. This is a terrible, horrible thing, brought on partially by my own incompetence and unassertiveness. Other people shouldn't have to learn my needed life lessons with me. --Walky

==What we've actually lost==
Okay, I've been trying to assess and give a realistic account of the information we appear to be missing. It's hampered by the lack of a search option, meaing I've been having to hit the random button until I've gotten to a page I can link to, and I may be missing something things, but I believe that we have lost the following sections:
*The work McFeely did on [[Energon (cartoon)|Energon]].
*[[Revelation]]
*[[All Hail Megatron]]
*[[The Reign of Starscream]]
*[[Maximum Dinobots]]
*[[Alliance]]
*[[Defiance]]
*[[Universe (2008)]]
*[[Revenge of the Fallen]] updates.
*Abates screen captures on [[Beast Wars]], [[Beast Machines]], and [[Generation 1]].
*Updated screen captures for [[Transformers Animated]] season 1 and 2.
*Last but not least, all we know of [[Transformers Animated]] season 3.

If I missed anything that stands out, please let me know. If all I've done is seriously depress you, then...I don't really know how to respond to that. -- SFH 13:49, 16 March 2009 (EDT)

:I'm not sure how worthwhile this is. I could add that we've lost almost everything related to the S.T.A.R.S. continuity (including the meta-page with that name), but seriously, we could keep listing pages forever. Not to mention that probably a huge majority of the existing pages have lost a lot of updates. I don't think there's anything we rank-and-file editors can do until either Walky appears brandishing a "routine backup" or Scout et al. get the Google cache sorted out and implemented. At THAT point, maybe taking stock would be a good idea, but until then, let's just go outside and get some fresh air. Toss a Frisbee around. Call up an old friend. - '''Jackpot''' [[Special:Contributions/216.231.37.133|216.231.37.133]] 14:01, 16 March 2009 (EDT)

::Yeah, we need a systemic, all-encompassing, computer-generated list of every page, since pretty much every page has probably been edited since the split. (though for the record, my list includes 15-20 Energon eps and 20-30 rewritten subgroup pages.) Said list needs a checklist option or something. I dunno. I had this written more coherently before the save page thing crashed. Whee, are we having fun yet? -- '''repowers''' [[Special:Contributions/99.140.167.2|99.140.167.2]] 14:28, 16 March 2009 (EDT)
:I still have all those screen captures, so I can reupload them if necessary. Of course, the images are actually still on the server, but the out-of-date database doesn't know they're there. I grabbed a complete set of Beast Machines episode articles and a near-complete set of RID episode articles from various caches last night. Unfortunately the software I wrote for doing the same only grabbed 53 articles before it was throttled. --[[User:Abates|abates]] 14:47, 16 March 2009 (EDT)
::We lost all the Titan collections that Emvee and I did. They weren't cached yet, but oddly the main Titan page had a cached version with the links I made to the individual pages. Wierd. This loss hits me really hard since I didn't start editing until this winter and '''everything''' I've contributed is now potentially gone. Since the tfwiki is one of the few joys I've had lately, it makes me feel a bit useless now. --Bluestreak7 [[Special:Contributions/98.213.132.186|98.213.132.186]] 15:03, 16 March 2009 (EDT)
::''Of course, the images are actually still on the server, but the out-of-date database doesn't know they're there.''
::If that's the case, then perhaps they can be reinstated when/if Bookworm compiles a list of files on the server that are newer than a given date.--Apcog [[Special:Contributions/99.158.0.251|99.158.0.251]] 15:38, 16 March 2009 (EDT)
That'd be Rob's work on Energon, not mine. I've lost around half the Victory cartoon, and ''every single fucking Japanese character article I ever did that's not Shuta.'' I thought we were only back to post-move September status, but we've actually regressed to June. - '''McFeely''' [[Special:Contributions/81.158.171.149|81.158.171.149]] 15:17, 16 March 2009 (EDT)
:I just saved as many of the Masterforce character articles as I could from Yahoo and MSN's cache (Google throttled me again) --[[Special:Contributions/203.97.2.142|203.97.2.142]] 16:15, 16 March 2009 (EDT)

In the event that we can't find a more recent backup (and at this point, if server dude can find one from even as recently as January, I'd be overjoyed), do we at least still have the original dump we exported in September which doesn't have all the wikia links at the bottom? --[[Special:Contributions/203.97.2.142|203.97.2.142]] 16:11, 16 March 2009 (EDT)

== Anything the rank and file could do? ==

OK, so there's obviously alot of scrambling going on from all who can to recover all the massive chunks of what's been lost, my question: is it at all helpful for the rank and file occasional contributor to be doing any editting at this point? I mean, I'm sure there are a bunch of us who would be willing to edit and add here and there to fix up what small bits and pages we can. Or would such small additions and peicemeal fixes just get in the way of the eventual large scale repairs? --[[Special:Contributions/71.235.138.121|71.235.138.121]] 14:59, 16 March 2009 (EDT)

:We really don't know yet. It depends on what Derik and Scout come back with. I can't imagine there's any harm in restoring pages by hand (it could presumably get lost in whatever mass upgrades we end up doing... or not. I have no idea.) Mostly, just sit tight for the moment. When we get a system figured out, we'll ring the bells to summon the fire brigade. -- '''repowers''' [[Special:Contributions/99.140.167.2|99.140.167.2]] 15:25, 16 March 2009 (EDT)

Pick a page. Stick <nowiki>{{bookworm}}</nowiki> at the top, it'll give you a link to that page's google cache. -[[User:Derik|Derik]] 16:33, 16 March 2009 (EDT)

== How do I properly add info to an article's "recover" page? ==

I've done the bookworm tag on some articles and saved the google cache to my harddrive... question is, how do I post this data to the articles' /recover sections? Just copy'n'paste the text, or go nab the source code? Detour --[[Special:Contributions/70.26.248.28|70.26.248.28]] 18:22, 16 March 2009 (EDT)
:I expect source code would be more useful. It will preserve the formatting, and someone can probably write a script to convert it back to wiki code. Plus, if I'm wrong, it's trivial to convert the source code to text, whereas you can't really go the other way. '''Chip''' [[Special:Contributions/69.139.147.179|69.139.147.179]] 19:01, 16 March 2009 (EDT)

:I do both, one after the other.
:The first is easier to re-format, but you'll often need to be able tog et the links out of the source code. -[[User:Derik|Derik]] 19:01, 16 March 2009 (EDT)

== Automated Cache Recovery ==
Will this program I happened upon help any? [http://warrick.cs.odu.edu/warrick.html Warrick]. It's supposed to be something that can automate getting websites from various web caches. I put a job in their queue, but if someone who has web stuff (it needs Perl 5 installed) set up on their comp can run the program themselves right now, that would help. Or is that the sort of thing Derik and Suki are already doing? (Basically, I'd like to help, but I don't even know what "phase" we're on...) --[[Special:Contributions/96.240.212.69|96.240.212.69]] 18:59, 16 March 2009 (EDT)
:It's being run. It'll take some time, and e3ven then... that's just grabbing a copy of the google cache before it vanishes.
:(Personally I'm banking on a localized reversal of [[The Fallen|the second law of thermodynamics]].) -[[User:Derik|Derik]] 19:01, 16 March 2009 (EDT)
::Well, if we're in the midst of saving the Google cache, than what we should do is... 1. Get a copy of as many cached pages as we can. 2. run them all through some sort of preprocessor 3. get a list of editors willing to help, give them each a chunk of the processed pages, and tell them to go to town reposting them. There's no sense in having people running around doing things piecemeal instead of being systematic about it.
::Which is why I was wondering if there was some sort of system already for having people grab cache pages, rather than just doing it randomly. I've been using [http://miscfile.alienharmony.com/transformers/allpages.htm this page] based on Abates' list to grab cache links for the Main namespace, and I have the first 400 links opened so far. Maybe if everyone took so many pages? (Although I have the advantage that I can force change my IP every time I run into Google's annoying throttle wall.)
::Also, I was thinking that we might consider just simply making a fresh install of the wiki software with an empty database, then simply recreating everything, if we're going to have to repost everything by hand anyway... --Jeysie [[Special:Contributions/96.233.179.58|96.233.179.58]] 20:48, 16 March 2009 (EDT)
:::Make that the first 500, which I'm going to work on saving now. Do our various editors want to post here and lay a claim on a set of numbers on the list and work on opening and saving them? (They link straight to the cache page.) Not all of the links converted cleanly, but it's probably easier to just manually fix them as you go instead of me trying to spend time manually cleaning and reposting the list. I figure, once we know we've got the Main namespace pages saved, we can then work on trying to figure out the other namespaces. (Template and Help at the very least.) --Jeysie [[Special:Contributions/72.79.218.239|72.79.218.239]] 21:06, 16 March 2009 (EDT)

== Restoring account status ==

Is it better to recreate an account, possibly messing up the caches for one's User Contributions, Talk Page, etc., so uploading images and such is possible? Or should we just sit tight a bit longer on that aspect until more restoration is done?--Apcog [[Special:Contributions/99.158.0.251|99.158.0.251]] 19:24, 16 March 2009 (EDT)
:Re-create. -[[Special:Contributions/207.224.69.82|207.224.69.82]] 20:02, 16 March 2009 (EDT)
::Feh. Okay, done.--[[User:Apcog|Apcog]] 20:13, 16 March 2009 (EDT)

== Pages "recovered" list ==

In the however likely case we are not able to get a full set of backups from Bookworm, we should probably keep a list of pages somewhere that have been at least partially recovered via Derik's bookworm template and google's cache or other caches. This way any rank-n-file editor who wants to help out can just look at the list to see what's been done so they can go save other pages. Another way is to get that listing of all the pages and have users check them off when they've been partially recovered. Thoughts? --Bluestreak7 [[Special:Contributions/98.213.132.186|98.213.132.186]] 19:53, 16 March 2009 (EDT)
:Well, here's a list of all pages that need to be ''recovered'' from the Main namespace: http://miscfile.alienharmony.com/transformers/allpages.htm with links straight to the cache pages. I've opened the first 500 links so far, and am half done with saving them, then I'll work on the next 500. Anyone else want to pitch in, lay claim to a range of numbers to save, and get on it? (Note: Not all of the links coverted over right, though most of them did. If you get an error, just fix any goofs in the link and try again. And keep a list of any pages where, even after checking the link was correct, it just didn't have a cache to save.) --Jeysie [[Special:Contributions/96.240.208.203|96.240.208.203]] 21:48, 16 March 2009 (EDT)

MediaWiki - User contributions [en]

MediaWiki talk:Community Portal/Damage Control Central