MediaWiki talk:Community Portal/Damage Control Central

From MediaWiki
Revision as of 15:25, 30 March 2009 by 81.108.237.26 (talk) (→Current status of the wiki: new section)
Jump to navigationJump to search

What happened
TFWiki.net's database was accidentally deleted in an attempt to upgrade the wiki-software. And due to a different software failure, the only existing backup was from June 2008. So one day we all logged in and saw that every article was nine months out of date. Our response was to raid Google's cache for more recent versions. The vast majority of articles have now been restored from there, but the Google-caching process has been an imperfect solution: Many articles are now up-to-date but badly-formatted, with garbled lines of code in plain view.

What needs to be done
The ugliness needs to be fixed, and some articles still need to be restored from Google's cache. Some of the work can be automated, but much will have to be done by hand. See the "Restoration" section below for resources and advice.

Information outside of the wiki
Immediately after the crash, discussion sprung up on these Allspark threads, and they have been a continuing source of information (especially as the wiki has been experiencing a lot of downtime lately):


Damage Control Central Archives
Archive 1 | Archive 2





Restoration

Basically, if you see an article that looks messed-up, feel free to fix it.

Useful Resources

Files

If an article is still in its June 2008 state, there is probably a newer version that it should be restored to. Here are places to look:

  • The Google cache. Add the name of the page you're looking for to the URL. (Don't click on the links in these pages; those will send you back here.)

As time goes on, the Google cache becomes less relevant because it overwrites itself with the current state of the articles. So several editors downloaded swaths of articles from the cache immediately after the crash:

All of those files were converted by Google from wikicode to HTML. Since the wiki software doesn't understand most HTML, we have used some programs to try to convert the files back. Here are the results:

And here's a program to open all of these archives:

HTML-to-wikicode processors

Here are the programs we've been using to convert the files back to wiki-format:

Fixing ugly pages

The simplest thing to do when you see a page with broken code all over it is to call it out. Put the Bookworm template at the top of the page, adding a severity-number and preferably a description of what's wrong. This will put the page in a category so everyone can see that it needs to be fixed.

If you'd like to help repair the damage, you'll find that most of the problems with HTML failing to convert back to wikicode have to do with templates. Oftentimes a dozen lines of HTML can be replaced with a single "{{stub}}" or "{{picsneeded}}". Some templates are more complicated, however. Those are explained in the next section.

The other big problem is references. These need to be re-created by hand via putting empty <ref></ref> tags in the proper places in the article, then picking out the actual reference from under the Footnotes/References section and sticking them in the Ref tags, then replacing the leftover messed-up code with the <references/> element. You may want to check against the unprocessed cache page to make sure it's all correct.

Template Code You Might Need

Breaking Lists Into Columns
{{columnlist|(number of items per column)|
(list of items)}}
Comic Issues

Items between "coverdate" and "continuity" should be customized to match the issue's actual credits and wikilinked to the person's name. 

{{Comicstory|
|seriesissue=(wikilinked comic series the issue is part of)
|prev=(previous issue)
|next=(next issue)
|seriesissue2=(wikilinked second comic series the issue is also part of, if applicable)
|prev2=(previous issue in the second series, if applicable)
|next2=(next issue in the second series, if applicable)
|title=(title of the issue, if applicable)
|image=(cover image)
|caption=(caption for the cover image)
|publisher=(wikilink to the publisher)
|date=(month, day, and year the issue was released)
|coverdate=(month and year on the issue itself)
|writer=
|penciler=
|inker=
|colorist=
|letterer=
|editor=
|continuity=(wikilink to the continuity the issue takes place in)
}}
Episodes

Items between "production company" and "animation studio" should be customized to match the episode's actual opening credits and wikilinked to the person's name. 

{{episode|
|series=(series the episode is in (codes: g1, headmasters, masterforce, victory,
zone, g2, beast wars ii, beast wars neo, beast machines, rid, armada, energon,
cybertron, animated))
|ep=(the episode number)
|prev=(previous episode)
|next=(next episode)
|series2=(second series the episode is in, if applicable (codes: g1, headmasters,
masterforce, victory, zone, g2, beast wars ii, beast wars neo,
beast machines, rid, armada, energon, cybertron, animated))
|ep2=(the second series episode number, if applicable)
|prev2=(previous episode in the second series, if applicable)
|next2=(next episode in the second series, if applicable)
|title=(title of the episode)
|japanese=(title in Japanese for Japanese-origin episodes, if applicable)
|romaji=(romanization of Japanese title)
|translation=(translation of Japanese title)
|image=(some representative image)
|caption=(caption for the image)
|production code=(episode's production code, if known)
|production company=(wikilink to the company that produced the episode)
|writer=
|director=
|animation studio=(wikilink to the studio that animated the episode)
|airdate=(the date the episode aired)
|continuity=(wikilink to the continuity the episode takes place in)
}}

First column is "Autobots", second is "Decepticons", third is "Humans", fourth is "Others", fifth is "Misc." These headers can be changed if needed by using |h1=(new header)|c1= in place of the c1=. Code sections with no characters in them can be deleted. 

{{featuredcharacters
|c1=

|c2=

|c3=

|c4=

|c5=

}}
"For Further Information"

Used whenever an article has had some of its sections separated into separate articles.
{{see|(title of separated section page)}}

Image Galleries

Used to format lots of images into a swanky-looking table. 

<gallery>
Image:(filename)|(caption)
</gallery>
"See Main Article"

Whenever an article has had some of its sections separated into separate articles, this should be placed at the top of the separate articles to link them back to the page they were split from.
{{main|(title of main article)}}

Specifying How an Article Should Be Sorted in Categories

For instance, if the article is named "Simon Furman", but you want it sorted as "Furman, Simon" in all categories.
{{DEFAULTSORT:(altered title)}}

Speedy Delete

Slap this on any page that can definitely be deleted. Needless to say, don't stick this on a page unless you're absolutely certain it can be deleted!
{{speedy|(reason for delete)}}

Stubs
  • General: {{stub}}
  • Character pages: {{charstub}}
    • Character pages that need fiction: {{charstubfiction}}
    • Character pages that need toys: {{charstubtoys}}
  • Comic issues: {{issuestub}}
  • Episodes: {{epstub}}
Year Page Navigation

Puts a nice navigation box on the top of year pages.
{{yearofthe|(year)}}

Google Cache Pages

Saved

  • Main
  • Template
  • Help
  • Transformers Wiki
  • Talk 0000-0499, 8600-8921

Exceptions

Not Saved

  • Talk 0500-8599
  • Template talk
  • Help talk
  • Transformers Wiki talk (non-Community Portal pages)

New Discussion

Probably the wrong place, but I have to ask this of those that handle coding/templates: Is it possible to fix the Recent Changes page to have the "View (newer 500) (older 500)" usable again? Just a thought. --Lonegamer78 22:18, 20 March 2009 (EDT)

So... how does this work? Is it basically if I see a garbled page I should feel free to clean it up? - Starfield 00:02, 21 March 2009 (EDT)

Pretty much. If you see something that's really bad, (like the code's all & lt; and & gt; and stuff), try downloading the zip file of unprocessed cache files, going to Abates' converter, choosing the appropriate cache file, and running it through again. That should make it a little more managable. --Jeysie 00:48, 21 March 2009 (EDT)

Lost articles

I ran my updated script over all of the cache files, so I could restore the ones which didn't update properly the first time. However some of the cache files are blank. Crossed out ones I have redundant copies of.

There were a couple of others, such as the Q&A which are already on the site, and I'm not sure which of the several "Heroic Legend" articles that one file is for, but perhaps someone has a copy of the other three? --abates 18:20, 20 March 2009 (EDT)

The big unprocessed cache zip has:
  • Heroic Legend: 2010 Wars
  • Heroic Legend: Head On, Master Warriors!
  • Heroic Legend: Optimus Prime VS Megatron!!
Also, considering that the "proper" Generation 2 article is a redirect, and it looks like we have all of the other main G2 articles, I don't think we need to worry about that one. --Jeysie 18:50, 20 March 2009 (EDT)
Cool! And I just grabbed a copy of Destined Confrontation out of MSN's cache, so consider this section closed! --abates 20:54, 20 March 2009 (EDT)

I have no idea where to ask about this

Are we able to rescue stuff from namespace pages, like sandboxes? I had an article for Bata, maker of licensed transformer footwear, almost raedy to go before the thing. Hooper_X 09:38, 21 March 2009 (EDT)

I was able to salvage the sandboxes that I made links to on my userpage-- because when google spidered my userpage, it saw links to them and spidered them.
If not... -Derik 19:41, 22 March 2009 (EDT)
That's about what I figured. Oh well. GOODBYE, SANDBALLS. Hooper_X 16:57, 23 March 2009 (EDT)

Template Help

So the second series on "A Heroic Battle" just isn't showing up. Did I do the code wrong (although I could swear it looks right), or is the template still a little messed up?

Never mind, I figured it out, although I should note that RiD, at least, doesn't seem to have the auto-prev and auto-next set up. --Jeysie 01:43, 21 March 2009 (EDT)

Also, I'm still curious about the Wookiepedia template... should I just erase it if I come across it, for now? --Jeysie 01:22, 21 March 2009 (EDT)

I see some of the spiffy new templates aren't working, like the new series of nameless characters templates (noname-nickname, etc.) and the faction icons. When I fix a page, should I restore those with the anticipation that the templates will return? - Starfield 12:34, 21 March 2009 (EDT)

That's what I've been doing. At least with the factions one (hopefully I got the faction names right!) Unfortunately I wasn't that familiar with the new nameless character templates, so I've been leaving them as "noname", and I guess once the templates have been sorted out, they'll be easy to spot by looking at the pages which link to noname. --abates 23:20, 21 March 2009 (EDT)
The big tar.gz has the "Nameless Characters" policy in it, as well as many other Help and Transformers Wiki files. I've been finding it useful for templates sometimes. --Jeysie 17:51, 23 March 2009 (EDT)

When a page's code is totally effed...

I've discovered that when a page's code is totally effed by the import script, it can usually be re-condition by running it through my page filter script. (located here)

The trick is to hit "wickify" not once, but TWICE. The second round generally sorts out most of the effed-uppingness. -Derik 12:38, 21 March 2009 (EDT)

I tried that with Student and nothing changed. Are there different kinds of effed-ness? Thylacine 2000--74.73.131.210 14:11, 21 March 2009 (EDT)
I accuse you of doing it wrong. It made a signifigant difference when I did it.
  1. Open the student page in editing mode.
  2. Copy the text out.
  3. Paste the text into the top text field of the Rewiki Text Page
  4. Click "Wikify"
  5. Click "Wikify" again
  6. Copy the text back out of the bottom text field, and put it into the student article.
  7. Preview

When I did this, Student went from being a total mess to something with sections, working links and a garbled skeleton that was clearly a notice box at the top of the page. -Derik 14:41, 21 March 2009 (EDT)

The big sweep

Okay- Deceptitran has a list of ~24,000 pages the wiki either has or thinks it OUGHT to have. (There's holes since this was compiled from imperfect lists, but it's what we've got.)

I'm getting ready to start coding a slow sweep over all the pages-- this will take several DAYS. The obvious edit to make is the removal of all the wikia links (which google has begun to notice.) Can anyone think of any other simple edits that should be added in that mix? (I want to be conservative and not fuck up the code even MORE than it already is.)

I'm probably gonna run a passive (no edits, much faster) sweep first and just try to figure out what pages are missing. -12:52, 21 March 2009 (EDT)

Big sweep is ~2% complete. Loss rate is around 1%. Good news-- most of the "lost" pages are false-positives. (Crap that got onto our paster page-lists by accidents.) actual flat-out missing pages seems closer to ~0.3%.
(Of course, the estimate of what's missing is only as good as our original lists. And if they were on the original lists-- then they should have gotten backed up and restored, right?)
It's something anyway. I'll post an actual list when the sweep finishes. -Derik 13:23, 21 March 2009 (EDT)

Excellent. Excellent excellent. There has just been waaaay too much screwed up stuff to take it on by hand. This is good to hear. -- Repowers 14:28, 21 March 2009 (EDT)

Can anyone think of any other simple edits that should be added in that mix?

Perhaps, for the sake of argument, replace &lt; and &gt; with < and >? I realise the result wouldn't be ideal, but it would paper over a lot of cracks in one fell swoop. - SanityOrMadness 18:47, 21 March 2009 (EDT)
Getting the last of the Wikia links? I'm still finding 'em.--RosicrucianTalk 19:31, 21 March 2009 (EDT)
I second that both removing Wikia links and converting &lt; and &gt; are the two big ones. Converting over the simpler/more standard templates would also be a plus, but I'm not sure how possible that is without risking further mangling. --Jeysie 22:00, 21 March 2009 (EDT)
Any line which looks like this:
<div style="border: 2px solid rgb(238, 238, 238); margin: 0pt auto 0.2em; padding: 2px; width: 90%; background-color: rgb(239, 239, 239); text-align: left;">'''Note:''' ''...''</div>
...can be turned into {{note|...}}. There are a lot of these. --abates 23:50, 21 March 2009 (EDT)
That nice-- except that's not what the Note template actually LOOKS LIKE int he page text. Either it's puer HTML, or the HTML's been escaped or... something, because Deceptitran isn't finding any instances of it when I run tests.
Can someone find me an example page with the screwed-up "Note" template on it (currently or just in history) so I can figure out how to code a fix? -Derik 19:54, 22 March 2009 (EDT)
My fixes for Movie Megatron's page
For instance: &lt;div style="width: 90%; margin: 0 auto .2em auto; background-color:#efefef; border: 2px solid #eeeeee; padding: 2px; text-align: left;"&gt;'''Note:''' ''[[Defiance]] changes the past relationship between Megatron and Optimus Prime as previously seen in IDW's Prime Directive prequel where they ruled side by side as equals. As presented here, Defiance establishes Optimus Prime as merely the leader of the Cybertronian science sector, and an underling to Megatron.''&lt;/div&gt; --Jeysie 20:42, 22 March 2009 (EDT)

Sweep results

*yawn* Good nap. 227 missing pages-- some of which is junk and not actually missing, some of which is stuck on Markons, etc. I see that some of this has been reccover while I slept, yay!

  1. '
  2. "
  3. (
  4. Ireneusz Załóg
  5. Jörmungandr
  6. Pick-Up (Bat-Robô)
  7. Rio Gráfica Editora
  8. 1
  9. 1984_media
  10. 2
  11. 3
  12. 4
  13. 6
  14. 9
  15. A
  16. All Fall Down (disambiguation)
  17. Banjō Ginga
  18. Binbōgami
  19. Būpink
  20. Bumblespud
  21. C
  22. Charlie Bodin
  23. Cheyne
  24. Chōkon Power
  25. Crown
  26. Cybertron (planet)/Gallery
  27. Cybertron Redux
  28. Daisuke Gōri
  29. Dark Designs (Titan)
  30. Destined Confrontation: The Children of Good and Evil
  31. Do Over
  32. Doryū
  33. DreamMix TV World Fighters
  34. Dreamwave Generation One continuity
  35. Drench (Shattered Glass)
  36. Drift (episode)
  37. Drill Bit (Universe)
  38. Dropbox
  39. Dropshot (Shattered Glass)
  40. Dublin James
  41. Duel in the Labyrinth
  42. Dungeons & Dinobots
  43. Durguth
  44. Dutch
  45. Dynamo
  46. E
  47. Each One Fights...
  48. Echo Across the Galaxy! Bell of Love!!
  49. Empire
  50. End of the Maximals!?
  51. End of the Road (Titan)
  52. Enter, Lio Convoy Typhoon
  53. Esther Scott
  54. F
  55. Fight! Super Robot Lifeform Transformers: The Comics
  56. Frederic Doss
  57. Fushigi Yamada
  58. Gairyū
  59. Geonosis
  60. Glen's grandmother
  61. Gōryū
  62. Grimlock (G1) toys
  63. H
  64. Hasbro Q & A
  65. Hasbro Q&A December 2008: Answers
  66. Hasbro Q&A December 2008: Question submission
  67. Hasbro Q&A March 2009: Answers
  68. Hasbro Q&A March 2009: Question submission
  69. Hitoshizuku Amaō
  70. Hōchū Ōtsuka
  71. Hoshinochō
  72. Hozumi Gōda
  73. I
  74. Image:Autobots+Lets+Make+Out-1836.jpg
  75. Image:Autobots+Lets+Make+Out-6341.jpg
  76. Image:Autobots+Lets+Make+Out-7270.jpg
  77. Image:BergerInc_Tanks&SolarPlant.jpg
  78. Image:Blasterbluesop&mdance.jpg
  79. Image:Coredevastator&prime.jpg
  80. Image:Coremegs&wheeljack.jpg
  81. Image:Dinobotislandratch&spark.jpg
  82. Image:G1_-_Fastlane_&_Cloudraker_-_Boxart.jpg
  83. Image:Gambitastro&crystals.jpg
  84. Image:Gambitastrotrain&scream.jpg
  85. Image:Gambittitans&cosmos.jpg
  86. Image:Insectsyndromeprime&megs.jpg
  87. Image:Jose+santacruz+menor.jpg
  88. Image:Maketrackshoist&huffer.jpg
  89. Image:Nergill_&_Troops.jpg
  90. Image:Omegasupreme&creature.jpg
  91. Image:Search4atelita&prime.jpg
  92. Image:Tfalost&foundbulkheadhocky.gif
  93. Image:Tud2bee&spikeembrace.jpg
  94. Image:Universe_Soundwave&Space-Case_toy.jpg
  95. Industrial Light & Magic
  96. Insecticon Attack!
  97. J
  98. J. Falconer
  99. Jirō Saitō
  100. K
  101. Kakuryū
  102. Kazuhiko Gōdo
  103. Kenichirō Tanabe
  104. Kensō Katō
  105. Kenyū Horiuchi
  106. Kōhei Kowada
  107. Kōichi Tōchika
  108. Kōji Totani
  109. Kōji Yusa
  110. Kōki Kataoka
  111. Kōmoribreast
  112. Kōzō Shioya
  113. L
  114. Lloyd Goldfine
  115. M.A.R.B.
  116. Mantarō Iwao
  117. Masaharu Satō
  118. Masashi Endō
  119. Matrix Quest (Titan)
  120. Matrix Quest(Titan)
  121. May 26
  122. Maya Klayn
  123. Meltdown's Experiments
  124. Mini-Vehicle
  125. Mirosław_Neinert
  126. Momotarō
  127. Monster Maezuka
  128. Mujō
  129. Naoki Tatsuta
  130. Nobuyuki Saitō
  131. Odette Yustman
  132. Off-Road Cycle
  133. Ohio
  134. Omega Sentinel (disambiguation)
  135. Omega Terminus
  136. Onslaught (ROTF)
  137. Open Fire!
  138. Optimus (episode)
  139. Optimus Prime (Armada) toys
  140. Optimus Prime (Movie) toys
  141. Optimus Prime cookie jar
  142. Optimus Prime Oral Care Station
  143. Osamu Saka
  144. Overbite (Shattered Glass)
  145. Overlord - Terror of the Chōkon Tornado
  146. Overlord (rank)
  147. Owen Hurley
  148. P
  149. Pacific Ring of Fire
  150. Pack-in material
  151. Package art/Gallery
  152. Packaging
  153. Palace
  154. Panini
  155. Panini Armada issue 2
  156. Panini Armada issue 3
  157. Panini Armada issue 4
  158. Panini Armada issue 5
  159. Panini Armada issue 6
  160. Panini Armada issue 7
  161. Panini Armada issue 8
  162. Panini Armada issue 9
  163. Paper Magic Group
  164. Paradron communicator
  165. PARD
  166. Pīpō
  167. Press Release: Teletraan-1 Wikia moves to TFWiki.net
  168. Professor Gō
  169. Q
  170. Quick_Bow
  171. R
  172. Rairyū
  173. Ratchet's_EMP_generator
  174. Reverb_(Cybertron)
  175. Robot_Masters_(cartoon)
  176. Ryō Naitō
  177. Ryōichi Tanaka
  178. Ryōka Yuzuki
  179. Sanryō Odaka
  180. Save_the_Little_Girl!_The_Chōjin_Warriors,_the_Godmasters
  181. Scoop_(Shattered_Glass)
  182. Seizō Katō
  183. Shinichirō Miki
  184. Shizuku Amaō
  185. Shōhei Kohara
  186. Shōji Kawamori
  187. Shōki
  188. Showdown_(disambiguation)
  189. Shūta Gō
  190. Slaves_of_the_Insecticons
  191. Smith_(disambiguation)
  192. Staff_Sergeant_Tracy
  193. Street_Action_Mini-Con_Team_(Armada)
  194. Takurō Kitagawa
  195. Talk:Hot_Shot_(Unicron_Trilogy)
  196. Teiyū Ichiryūsai
  197. Teletran-1:_The_Transformers_Wiki:About
  198. Template-episodenav-dev
  199. Tesshō Genda
  200. The_Big_Book_of_Coloring_Fun
  201. The_New_Battle_Begins!
  202. The_Smashing_Pumpkins
  203. The_Treacherous_Attack_of_the_Decepticons
  204. Tight_Shot
  205. Toaster_bot
  206. Transformers_Mix_&_Match
  207. Transformers_Wiki_talk:Community_Portal/Cache_Recovery
  208. Transformers_Wiki:WikiaBotTests
  209. Transformers:_Binaltech_&_TF_Collection_Complete_Guide
  210. Trevor_Hutchison
  211. Ultra_Magnus_(Universe_Spy_Changer)
  212. Ultra_Magnus...to_the_Rescue?
  213. Units_of_time/Continuity
  214. User:M_Sipher/Sandbox
  215. Vector
  216. Yōichi Kobiyama
  217. Yōji Ietomi
  218. Yōko Kawanami
  219. Yokuryū
  220. Yūgo Ōgami
  221. Yūji Kishi
  222. Yūji Mikimoto
  223. Yūki Ōshima
  224. Yūsaku Yara
  225. Yūto Kazama
  226. Yukiyoshi Ōhashi
  227. Yumi Tōma
I've recovered a bunch of those from Google's cache. Many of them are malformed because they have accents (I already went through and reuploaded as many of the accented articles as I could find). A chunk in the middle from Off-Road_Cycle to PARD are in G1MarvelBlaster's saved cache stuff. Mini-Vehicles is at Mini Vehicles, Ultra Magnus...to the Rescue? is at Ultra Magnus... to the Rescue?. I started trying to look for the remaining legit lost articles on yahoo and MSN, but I don't have any more time to work on it tonight. One of the message board people who saved stuff may have some of them? --abates 04:06, 22 March 2009 (EDT)

Filtering the sweep results

So, just from that list, once you take out (1) the links which have gone blue and (2) obvious errors of one sort or another (as well as the single-character stuff, the Teletran-1...About page was deleted after the big crash, f'rinstance. I stuck a {{speedy}} on it myself), that leaves these as actual pages which are completely missing, yes? [Not swearing none of these were redirects/etc - if they were, please send them to where they're meant to go and remove them from this list, huh? I've left the disambigs, tho.]

  1. Charlie Bodin (minor role in live-action movie - http://www.imdb.com/name/nm1835107/ )
  2. Cheyne (Unrecoverable. Yahoo doesn't have cache, but a search says "The planet Cheyne was one of several worlds explored by the Autobots ... Retrieved from "http://tfwiki.net/wiki/Cheyne" Categories: Generation 1 | Planets ...")
  3. Crown (unrecoverable)
  4. Digital Dagger (Unrecoverable. T.E.C.H. toy from first movie's toyline)
  5. End of the Maximals!?
  6. End of the Road (Titan) (Unrecoverable.)
  7. Esther Scott
  8. Frederic Doss
  9. Fushigi Yamada
  10. Geonosis
  11. Hasbro Q&A March 2009: Question submission
  12. Insecticon Attack!
  13. Kazuhiko Gōdo
  14. Lloyd Goldfine
  15. Matrix Quest (Titan) (Unrecoverable.)
  16. May 26 (Unrecoverable. Yahoo gives it as a result in a search, but no useful info)
  17. Maya Klayn
  18. Odette Yustman
  19. Robot Masters (cartoon)
  20. Scoop (Shattered Glass)
  21. Slaves of the Insecticons
  22. Smith (disambiguation)
  23. Staff Sergeant Tracy
  24. The Big Book of Coloring Fun
  25. The Smashing Pumpkins
  26. Tight Shot (Unrecoverable. T.E.C.H. toy from first movie's toyline)
  27. Ultra Magnus (Universe Spy Changer)
  28. Vector
  29. Yukiyoshi Ōhashi

63 pages as I type... obviously, this doesn't take account of stuff which exist but was reverted, but... - SanityOrMadness 16:42, 22 March 2009 (EDT)

There are certainly more-- this list was generated by from lists of pages from ABates and FortMax we think we're SUPPOSED to have. (and thus pages we were trying to save the cache of.) Literally this is more a compilation of- "Stuff we KNEW we had to save we either fucked up on, or there was no cache of."
If a page wasn't on the lists in the first place (for a variety of reasons) then it's not gonna be there. Sorta "Known Unknowns" vs. "Unknown Unknowns." -Derik 18:27, 22 March 2009 (EDT)
Yeah, I figured that out pretty quickly after I typed that when I noticed the number of redlinks on extant pages vs. blue links on cached pages, especially around the Japanese stuff (half of the Beast Wars Neo episode articles appear to be gone, for instance). - SanityOrMadness 18:33, 22 March 2009 (EDT)
Lonegamer has many (if not all, I dunno) of the Beast Wars Neo episodes in her saved caches, as well as lots of other Japanese article-related stuff, as I posted here earlier, for those folks who typically work on the Japanese articles. *hint*nudge*kick* --Jeysie 19:20, 22 March 2009 (EDT)
The vast majority of the Neo episode articles were just skeletons and are available on the Other Place. It's not really that big of a deal. —Interrobang 20:55, 22 March 2009 (EDT)
So what's the possibility of pulling old versions of these pages from the Wikia version, i.e. the current versions as of when we left? -- Dark T Zeratul 18:45, 23 March 2009 (EDT)
Those (currently) 30 pages? Virtually nil - they're all redlinks over there except End of the Maximals!?, which is a virtually-empty shell. - SanityOrMadness 19:57, 23 March 2009 (EDT)
ISTR I added Kazuhiko Gōdo and Yukiyoshi Ōhashi less than a week before the Event, so I'm not surprised they're unrecoverable. --abates 20:14, 23 March 2009 (EDT)

And so it Begins...

Deceptitraan has begun to do an edit sweeps. I'm calling this "Pass 1." He will ATTEMPT to;

  • Remove the wikia spam
  • Fix the {{factions}} template
  • properly format the {{disambig2}}'s
  • Fix {{note}}s

This is going to be at least fourteen thousand edits, minimum. It's gonna take awhile. (The 'what's missing' sweep was 20-100 times faster because it didn't have to post edits.) -Derik 23:11, 22 March 2009 (EDT)

While I think of it, I'm gonna add those filters to my page-fixer tool too. Let me knwo if that cause it to glitch. -Derik 23:21, 22 March 2009 (EDT)
On-the-fly change-- Deceptitran is now also fixing links like [[Slumdog]] to simply read [[Slumdog]]. (The import script rendered everything as a two-part link.)
I'll go back and re-check the 500-or-so articles already edited for this after. Note to self: The articles I want to re-check are the ones lacking flag2 in the database. -Derik 00:10, 23 March 2009 (EDT)
Three little points, based on a tiny sample of Deceptitran's edits:
  1. Careful which side of the [[x]] links you grab. The general rule seems to be that the left hand side always has the first letter capitalised, and I've seen a couple of pages where [[continuity family]] has become [[Continuity family]] when the "c" should have stayed small.
Just inadvertently confirmed, because Deceptitran filtered THIS page! (i.e., the "links" above". I reverted :)) It indeed took the capitalised left-hand side. - SanityOrMadness 01:13, 23 March 2009 (EDT)
  1. Can you grab the equivalent {{storylink}}s while you're at it? There's a lot of {{storylink|x|x}}s too for much the same reason.
  2. On Buzzsaw (Cybertron), the {{note}} initially got messed up. I think it's an edge case, since others seem to have gone through okay, but something to watch if you haven't fixed it.
Otherwise, keep up the good work :) *wonders aloud if this might fix the article-counting problem - if a page gets edited, or even null-edited, it gets readded to the numbering if it's slipped out, doesn't it?* - SanityOrMadness 00:25, 23 March 2009 (EDT)
What voodoo are you using to spot [[Slumdog]] links? I couldn't get that one worked out. --abates 02:37, 23 March 2009 (EDT)
I was using a callback-- /\[\[(.+)\|(\1)\]\]/i --> $2.
Find all links where the second term is the same as the first term, using case-insensitive match, and use the second term. (I was originally using the first, oops.)
Problem is I eventually realized that would turn links to [[Energon Cube|energon cube]] into [[energon cube]], and only the first letter can be insensitive. :p
Now I'm using a callback function. Like killing flies with a nuke. (And I'm gonna use this page to test that callback.) *sets up some bad links.* -Derik 03:42, 23 March 2009 (EDT)
Woohoo! It parsed 'em all right, we are good to go!
It looks like I ran about 700 pages through the bad version of the filter but... I have a hard time making myself care. That's about 1.2% of our pages, and 98% of the time the difference between the two types of links will be nil or cosmetic. It's right going forward, but I'm not going back to fix the others.
Also, I looked at the buzzsaw link you posted-- the {{note}} part got rewiki'd perfectly! It's just that it had a leftover HTML link in it!
There's like 50 variants of <a href=""> on the site right now because browsers tended to save the files as THEY thought they looked.
Open a page in firefox and hit "view source." Now highlight a chunk of a page and right-click "view source." THE CODE IS DIFFERENT. The right-click gives you the HTML as it's been shuffled around by the geko rendering engine, which puts all the classes, titles, hrefs and so on in the SAME ORDER, regardless of what their sequence was in the original HTML.
I figure there's gonna be some MONSTER link-parsing function that takes all of this into account for another sweep after this. :p (at which point the anchor tag within that perfectly valid {{note}{ template would be correctly wikified.)
There's nothing wrong with the {{note}} filter-- it's just that it only does {{note}}s. -Derik 03:57, 23 March 2009 (EDT)
FYI, the "BIG SWEEP" is just under 20% complete. About 4000 pages has been edited by Deceptitran, removing wikia spam, fixing links and a couple common templates. Hit "show bots" in recent changes to see him in action! -Derik 17:42, 23 March 2009 (EDT)

Lonegamer's Cache

Lonegamer sent me all of her browser cache files the other day. I've already reposted a few I can recognize as being ones we lost, but there's some other possibly useful stuff in here, lots of Japanese episode and manga-related stuff, especially. I link to it for those who might need such things: http://miscfile.alienharmony.com/transformers/lonegamers-cache.zip --Jeysie 19:30, 21 March 2009 (EDT)

Fantastic! I was able to use some of the files from here to fix a bunch of articles. --abates 19:33, 22 March 2009 (EDT)

G1MarvelBlaster's cache saves

It doesn't look like these were part of the original upload of files: [1]. I can't seem to download the files though. --abates 01:19, 22 March 2009 (EDT)

I just downloaded Omega bomb cache article. Seems to be working alright. Is there some reason why people decided to not upload the files to scout's server so they could be restored en-mass? It seems to me that some people deciding to upload files elsewhere is the reason for so many articles not being updated. --FFN 01:41, 22 March 2009 (EDT)
I know there was a problem with Scout's FTP being down at one point, so it's possible people decided to upload elsewhere during that time. (I personally had just uploaded the pages right after saving as a backup, before Scout set up her FTP, but I offered to reupload if needed... no one told me to, however.)
I will note that G1MarvelBlaster and other folks linked to their uploads in the original finish summary, though, so they did inform that they uploaded it somewhere else. --Jeysie 02:04, 22 March 2009 (EDT)
I managed to get Paradron communicator out by hitting download, copying and pasting the HTML code into a file, saving the file, and then processing it. --abates 02:38, 22 March 2009 (EDT)
So does someone want to volunteer to grab these and reprocess them? You can skip all the October files, since I think Derek did those, and several of the Optimus Prime ones have already been done. Should be around 160 in total. I'd do it, but I have 600+ articles which are messes to try to reprocess. --abates 16:23, 22 March 2009 (EDT)
I'm starting to work on these. I'm currently running them through your conversion script and then pasting the code into the Wiki page. Is this the fastest method?--Tigerpaw28 14:26, 25 March 2009 (EDT)

Zip file

Here is a 1MB zip file containing 131 of the cache files. I've left out the ones which I know for sure have already been updated. --abates 01:16, 23 March 2009 (EDT)

I've tried to download it, but what I get is only 403 bytes and contains no files. - Jackpot 14:31, 25 March 2009 (EDT)
His link was incorrect. He had a b instead of a v in G1Marvelblaster. The link is fixed now and I've already done the first 10 files in the zip. --Tigerpaw28 18:32, 25 March 2009 (EDT)
D'oh! Thanks for that! --abates 18:55, 25 March 2009 (EDT)
Thanks! It works fine for me now. - Jackpot 19:22, 25 March 2009 (EDT)
No problemo. Updating my status as having the first 20 65 files complete (Octopunch to Omega_Supreme_(Animated) Origin). Yay, halfway there.--Tigerpaw28 21:55, 25 March 2009 (EDT)

Why are we blocking the Wayback Machine?

I accidentally hit the Wayback Machine rather than MSN when I was trying to grab the cache for a page there, and was informed that:

We're sorry, access to http://tfwiki.net/wiki/End_of_the_Road_(Titan) has been blocked by the site owner via robots.txt.

Wha-huh?! - SanityOrMadness 10:34, 22 March 2009 (EDT)

There was nothing in our robots.txt that should stop the Internet Archive from viewing us, but there also wasn't anything useful in it anyway, so I've deleted the file. That's weird. --Suki Brits 12:02, 22 March 2009 (EDT)
It's probably Archive.org's default answer-string for "I don't have it."
Just like YouTube video player informs you a video is no longer available even if it it's just having a timeout error. -Derik 18:22, 22 March 2009 (EDT)
No, the default is simply "Sorry, no matches" (example) - SanityOrMadness 21:01, 22 March 2009 (EDT)
I believe the Wayback Machine's crawlers simply look for the presence of a robots.txt and not the content of it. At least that's the impression I've gotten from using the Wayback Machine.--Tigerpaw28 13:15, 23 March 2009 (EDT)

Page shrinkage

I took a stab at cleaning up the loose code on Galvatron (G1), restoring quotes and notes and whatnot, and now the page seems to be half its normal width. Uh, help? I mentioned this in the Summary box too, but for some reason it didn't appear in Recent Changes. --Thylacine 2000 14:17, 22 March 2009 (EDT)

Too big to wikify?

I have attempted to convert Optimus Prime (Armada) a few times, as recovering that will let us split the toy section out again. It seems to choke Derik's tool, though.--RosicrucianTalk 18:52, 22 March 2009 (EDT)

Do it section-by-section. Derik's tool DOES have an upper limit, but if you put it through in smaller chunks, it'll work. - SanityOrMadness 18:56, 22 March 2009 (EDT)
I've restored the toy page from G1MarvelBlaster's cache version. I'd suggest getting Optimus Prime (Armada) from there too, as it looks like that might be a more up-to-date version than was originally imported here. --abates 20:29, 22 March 2009 (EDT)

Incidentally, Optimus Prime (Armada) toys is still in Google's cache in its' own right. - SanityOrMadness 19:27, 22 March 2009 (EDT)

It's in G1MarvelBlaster's archive of cache files too. --abates 19:36, 22 March 2009 (EDT)
Yeah, my took is kinda hacky. I didn't know about the limit... but it doesn't surprise me. I've got an AWFUL LOT of "match everything" Regex's set to Multiline mode, with callbacks. Enough KB of text probably cause that to choke. -Derik 19:38, 22 March 2009 (EDT)

I noticed Sanity or Madness pulling his hair out over all the garbled talk pages-- both too long for my tool and indented which it doesn't handle.

Well it handles indents now. And especially or talk pages Version 3i. It just does indents and headers, so it can filter long pages. (You'll still have to break 'em up to fix everything else, but at least now they're more comprehensible when you try to.) -Derik 03:18, 23 March 2009 (EDT)

Working on scrambled pages

Some of the pages ended up completely scrambled - IE no HTML conversion and, you'll note, no categories. I've been restoring as many of them as I can, but some of the original cache files have really strange line breaks in them, often in the middle of HTML tags and both mine and Derik's scripts pretty much choke on them. I went through the articles on my list which start with the letter 'D' tonight, saving new copies of them from Google, Yahoo and MSN, however I couldn't recover these four:

So if anyone has spare copies of these, please speak up! :) --abates 04:46, 23 March 2009 (EDT)

Done, I think. --Jeysie 05:34, 23 March 2009 (EDT)
Woo! Cool! Having done a mass process of the rest of the files on my list, the following three are the only ones I couldn't retrieve from caches:
It looks like the other 510 files processed all right, so now I just have to go through them all and paste them into the articles on teh Wiki --abates 05:45, 23 March 2009 (EDT)
What did you code your tool in anyway? Do you want copies of my PHP callbacks?
I really think Sweep #2 is just going to require a "Find an <a> tag, now pick it apart and put it back together no matter what order the fucker is in." function. Admittedly, this is more-or-less how I wrote the indenter-- it's a progressive parser, just like the Bad Old Days of write-it-yourself XML parsers. -Derik 06:40, 23 March 2009 (EDT)
It's coded in Perl. Sure, a copy of the callbacks would be useful, thanks!
I never could get the indenting right - it doesn't help that the HTML MediaWiki generates doesn't seem to close its <li> tags all of the time. --abates 07:02, 23 March 2009 (EDT)
This is PHP not perl, but the basic concept applies...
function fixIndents($wiki_text){
	$lines = explode("\n", $wiki_text);
	$new_text = '';
		$options = array(	'down' => '<dl>',
							'up' => '</dl>',
							'new_sib' => '<dd>',
							'end_sib' => '</dd>'
						);

	$indent = 0;
	foreach($lines as $line){
		$evaluate_line = true;
		$offset = 0;
		while ($evaluate_line){
			$position = null;
			$mode = null;
			foreach ($options as $mode_type => $option){
				$loc = strpos($line, $option, $offset);  //I'm pretty sure $offset is always 0 when it hits this line.
				if ($loc !== false){
					if (($position === null) || ($loc < $position)){
						$position = $loc;
						$mode = $mode_type;
					}
				}
			}//End Options
			if ($position !== null){
				//echo "<p>$mode</p>";
				$offset = ($position);
				switch ($mode){
					case 'down':
						$indent++;
						$replacement = '';
						break;
					case 'up':
						$indent--;
						$replacement = '';
						break;
					case 'new_sib':
						$indent2 = $indent;
						if ($indent2 < 0) $indent2 = 0;
						$replacement = str_pad('',$indent2,':');
						break;
					case 'end_sib':
						$replacement = '';
						break;
				}
				$line = substr_replace( $line, $replacement, $offset, strlen($options[$mode]) );
				$offset=0;
			} else {
				$evaluate_line = false;
//				echo $line;
			}
		}//Endline
		//$problems = array();
		//$problems[strpos($line,$begin_parent)];
		$new_text .= $line;
	}
	$wiki_text = $new_text;
	$wiki_text = str_replace( '</p><p>', "\n\n", $wiki_text );
	$wiki_text = str_replace( '<p>', '', $wiki_text );
	$wiki_text = str_replace( "\n</p>", '', $wiki_text );
	return $wiki_text;
}

I break it up line-by-line just o to reduce overhead.

Then I scan for DL's and DD's, looking for which is "next." If I encounter a DL, I add one to the #of indents. (DL means you're going 'down' a level, <DL means you're going up one.) The DL and /DL tage themselves you can just erase then.

They each DD represents the start of a new line- which has to be indented with the correct number of :'s (which has been going up and down every time we wan into a DL or /DL.) So i replace the DD with, for example "::::" (four indents.) You can just erase the /DD's.

It sounds really stupid, but it works perfectly. My messy code is really just a reflection of the fact it has to be de-parsed using a state machine instead of stateless regular expressions. -Derik 09:37, 23 March 2009 (EDT)

Callbacks

preg_replace_callback('/<a href="(.+(#.+)?)"( class="(.+)")? title="(.+)"( rel="(.+)")?>(.+)<\/a>/Ui'  , 'fixLinks' );

function fixLinks($matches){
	$url = $matches[1];
	$anchor = $matches[2];
	$specialClass = $matches[4];
	$title = $matches[5];
	$prettyText = $matches[8];
	$returning = '';
	if (($title.$anchor) == $prettyText){
		$returning = "[[$prettyText]]";
	}else{
		//Text is different
		$returning = "[[$title$anchor|$prettyText]]";
	}

	if ($specialClass == 'external text'){
		$returning = "[$title $prettyText]";
	}
	return $returning;
}
/* ===========================================================================
   Note that this will still return [[Energon cube|energon cube]] because it doesn't
 treat the first character as caseless-- but the NEXT callback will fix that...
   This function will also correctly parse Wikipedia links, Wookiepedia links, and
 external links.  Yay!
=============================================================================== */

preg_replace_callback('/\[\[(.+)\|(\1)\]\]/iU'  , 'fixLinks2' ); //\1 is a mid-pattern callback to the first() subpattern.  "If the text is the same withotu case, refer the link to the callback function."

function fixLinks2 ($matches) {
	$one = $matches[1];
	$two = $matches[2];
	if ( substr($one,1) == substr($two,1)){
		return '[[' . $two . ']]';
	} else {
		return '[[' . $one . '|' . $two . ']]';
	}
} //Literally just compares substrings excluding the first character, LOL.

...all 3 of these came from, different script files, oddly enough. I don't have ONE script that does them all. ;)

Header callback

preg_replace_callback( '/<a name=".+"><\/a><h([123456])><span class="editsection">\[<a href=".+" title="Edit section:.+">edit<\/a>\]<\/span> <span class="mw-headline">(.+)<\/span><\/h[123456]>/Ui'  , 'fixHeaders');

function fixHeaders($matches){
	$padding = str_pad( '', $matches[1], '=');
	return $padding . $matches[2] . $padding . "";
} //How many ='s?  Just count the number on the tag!  1 for H1, 6 for H6.

Hope that's helpful. Is perl Ecmascript? -Derik 09:53, 23 March 2009 (EDT)

It's from the same family tree as PHP and Ecmascript. The syntax is very similar, in fact! --abates 15:55, 23 March 2009 (EDT)

Derik! In case it's helpful, here's my latest code for rewriting the links: 

sub fixlinks {
  my $str=shift;
  my $pos=0;
  while(substr($str,$pos) =~ /(<a ([^>]*?)>(.*?)<\/a>)/i) {
    $wholelink=$1;
    $builtlinkcode="";
    $linkcode=$2;
    $text=$3;
    $pos=index($str,$wholelink)+1;
    next if($linkcode =~ /name=/i);  # skip <a name=
    $class=""; $title=""; $url="";
    $class=$1 if($linkcode =~ /class="(.*?)"/i);
    $title=$1 if($linkcode =~ /title="(.*?)"/i);
    $url=$1 if($linkcode =~ /href="(.*?)"/i);
    if($class eq "extiw") {  # interwiki, usually to Wikipedia
      if($url =~ /\#(.*)$/) {
        $title.='#'.$1;
      }
      $builtlinkcode = "[[$title|$text]]";
    } else {
      if($class =~ /^external/i) {  #external link
        $builtlinkcode = "[$url]" if($class =~ /external autonumber/i);
        $builtlinkcode = "[$url $text]" if($class =~ /external text/i);
      } else {
        if($url =~ /\#(.*)$/) {
          $title.='#'.$1;
        }
        $firstletter=substr($text,0,1);
        $matching=((substr($title,1) eq substr($text,1))&&($title =~ /^$firstletter/));
        $title = ':'.$title if($title =~ /^(Image|Category)/i);
        $builtlinkcode="[[$title]]" if($matching);
        $builtlinkcode="[[$title|$text]]" if(!$matching);
      }
    }
    substr($str,index($str,$wholelink),length($wholelink))=$builtlinkcode;
  }
  $str;
}

No more [[boing|boing]] links! :P --abates 17:57, 23 March 2009 (EDT)

OK, not quite perfect yet, but it catches most of them! --abates 19:33, 23 March 2009 (EDT)

Faction template

Faction template is working, but someone (preferably not me,) needs to go in and fill in all the icons again or it's gonna start sprewing garbate all over page titles. Only a medium level of template competence is require,d it's all copy-and-paste.

  1. Go hereTemplate:Factions/icons.
  2. Starting with the Maximal/Predacon entries as a model, add all the other factions.

You can find all the names to use and the image files that go when them only slightly garbled in the table here; Template:Factions.

Just copy and paste as swap out the names. Volunteers? -Derik 12:44, 23 March 2009 (EDT)

Well, now I know why I've never seen it working... why DOES it require JavaScript? It isn't hard to specify an absolute position to place in the header using CSS alone, surely? - SanityOrMadness 12:52, 23 March 2009 (EDT)
<Raises hand> I'll do it! Shouldn't take me more than a couple hours, if that. --Tigerpaw28 13:57, 23 March 2009 (EDT)

And it's done. All the symbols in the list Derik linked to have been added to the template. I've verified that all the images are working with one exception: the Blendtron logo. I tried the filename listed on the linked list as well as the Blendtron page and neither shows up. I'm not sure if the image file listed for predaconrid is right or not. Is it supposed to be the same as the Beast Wars Predacon? Also, some of the mouseover text may need to be changed in order to restore Teh Funny. I can do that myself if someone can provide me with a list of what needs to be tweaked. --Tigerpaw28 16:32, 23 March 2009 (EDT)

I didn't think we had funny mouseovers on these things.
Thanks for the work, it was nice not to be attacking it on my own-- I'll take s look at the files to see if some are missing.
Answering Sanity||Madness's question... absolutely-positioned CSS is unideal because it can obscure long titles. Unlikely to happen with character names... but for some unnamed characters with descriptive titles it could happen. With Javascript everything is inline and will wrap correctly if it gets cramped. -Derik 20:21, 25 March 2009 (EDT)
You're welcome. The reason I ask about funny stuff is because of the articles/mouseover text present in the Template:Factions list. Specifically, Mutants links to 'Filthy muties', Cobra to 'Can the world oppose deadly Eskimos?' and Go-Bots to 'Go GO GO-BOTS!'. I couldn't tell if that was supposed to be part of the template or not, considering none of those pages exist.--Tigerpaw28 21:02, 25 March 2009 (EDT)

Bookworm template

I'd like to use Derik's Bookworm template to help organize the restoration process as I edit individual pages to varying degrees of completion. But I think a few improvements could go a long way toward making it more relevant, and for the sake of not cluttering this page up, I've suggested my ideas on its Talk page. - Jackpot 21:13, 23 March 2009 (EDT)

TBH, I'm not sure what the point of using that template is any more... why bother with stages instead of just going ahead and completely cleaning up any messed-up pages you come across? Saves everyone's time that way. --Jeysie 21:23, 23 March 2009 (EDT)
Hmm, I already have a list of all pages at "stage 1" (imported, but needs reconverting.), which I'm slowly getting though.
The tricky ones are those at "stage 0" and identifying which ones should have been updated but weren't (I noticed Money Is Everything wasn't). Sometimes these are pages which were moved between June last year and now, so the cache version has been uploaded at a different place (All the Reign of Starscream issues). Sometimes it was deleted between June and now.
So I suspect that if you come across what looks like a legit content page, and it has a Wikia link at the bottom, and you don't know why it hadn't been updated, it might be handy to slap a bookworm template on it? --abates 22:06, 23 March 2009 (EDT)
Makes sense to me. In answer to Jeysie's question, the value is that I don't always have the time to patiently untangle garbled code, but seldom am I so harried that I can't put a template on a page. And even if I do have a decent amount of time, the work isn't always worth it at the moment. For instance, I'm restoring Community Portal archives right now, and at first the convertor was messing up the thread-indentation. This was both a minor problem and a major amount of work to fix, so I didn't want to devote the time yet, but I did want to leave an obvious trail so it could be taken care of later when the time was right (or ignored in perpetuity if nobody ended up caring, but at least people would know the issue existed). The Bookworm template, with an option for a comment, would've been perfect. Now, all that having been said, this issue no longer exists, as I've discovered that the other convertor handles indentation fine. But it's hard for me to imagine that there are no other problems like this scattered throughout our thousands of pages. - Jackpot 22:36, 23 March 2009 (EDT)
I guess I figure... I'd rather just be patient and work on things one-by-one... if I come across a page that'll take a while I just dig into it and work on it in bits until I'm done with it, then move on. I'd rather just fix pages as I come across them instead of slapping a template on it and hoping someone else will do it.
But if it's a problem where someone really can't figure out the coding, then maybe some of us who are more wikicode-familiar can offer to help. I'm willing to have anyone who comes across a page they totally can't figure out dump a link to it on my talk page. --Jeysie 22:48, 23 March 2009 (EDT)
I agree with Jackpot -- when we come across an article that's just a trainwreck, we need a way to mark it as such. Frankly, I'm not prepared to take on a mess like Greg Sepelak's article, but I hate leaving it unmarked. A way to designate "needs major work" would be handy. -- Repowers 08:03, 24 March 2009 (EDT)
*cough* Done. I just ran the cache page back through Abates' converter, which did the lion's share of the work, tweaked what didn't get fixed, and posted it. --Jeysie 08:11, 24 March 2009 (EDT)
Addendum: Now, references... I will say that those suck. *stares at the 2K7 movie page with a sigh and puts it on her to-do list after some sleep* --Jeysie 08:22, 24 March 2009 (EDT)
I have a big list of the trainwrecks. Current count: 425 articles, all starting with the letters G, R, or S (LOTS beginning with S), and I've been working at it nightly. I expect to be finished sometime this weekend. :) --abates 15:39, 24 March 2009 (EDT)
So I suspect that if you come across what looks like a legit content page, and it has a Wikia link at the bottom, and you don't know why it hadn't been updated, it might be handy to slap a bookworm template on it?
I'm going to repeat this suggestion, because I've noticed a couple of incidents of people removing the wikia crap from pages, but the reason those pages weren't updated was because they were moved before the data loss and they exist under a different name. --abates 19:04, 24 March 2009 (EDT)
Still agreed, and on a slight tangent: We should encourage the immediate purging of any Wikia-link on sight, but that may be the only obvious indication that a page is massively outdated or occupying a mistranslated namespace or whatever. So rather than feel obligated to deduce the true nature of the page and restore it every time we remove that bread-crumb, we should feel free to slap a {{bookworm|0}} on it and move along. - Jackpot 20:31, 24 March 2009 (EDT)

To Anyone Who Works on Marvel US Pages Before I Do...

While double-checking links on the main Marvel US page, I noticed that some of the Marvel US pages seem to be sporting the original comicnav... which means they either didn't update properly or the Google cache was old, as I know I updated them all to the Comicstory template. Just an FYI. --Jeysie 00:58, 24 March 2009 (EDT)

Yesterday's Heroes!, for instance? I'm guessing it got uploaded at the wrong name and someone tagged it for deletion without moving the code over? Also Ca$h and Car-nage! is now an article about the Byacrane. --abates 07:44, 24 March 2009 (EDT)
In fairness, kids do need to learn more about Byacrane. -Derik 08:30, 24 March 2009 (EDT)

Macron issues

Currently, the Wiki is containing two pages of every article with a macron in its title. One with the macron properly over the letter, and one with the two characters separate. (I.e., the regular version has ō as G%C5%8 in the url. The fucked up version has it as o%CC%84.) I don't know how to handle this issue, since clicking on the title with the characters separate just takes you to the page with the characters together. Their individual diffs can be accessed, though. Perhaps somebody with a old browser that doesn't support the macron sign can fix it? —Interrobang 06:25, 24 March 2009 (EDT)

I don't think an older browser will make any difference - the redirect is occurring at the server end. --abates 07:35, 24 March 2009 (EDT)
Okay, that's annoying. I guess somebody with the ability will have to dig through the Wiki's drive to delete them. Scout, I guess? —Interrobang 08:22, 24 March 2009 (EDT)
Hmm, I wonder what would happen if you tried moving one of the affected pages (not that I'm suggesting anyone try this, at least until we have backups working).
My gut instinct is to say we should leave it as a problem until we're back up and running at 100%, since it doesn't seem to be stopping anything from working. --abates 16:00, 24 March 2009 (EDT)
I don't have a problem with prioritization; I just put this up so I wouldn't forget about it later. The one and only problem arising from them (other than taking up space) is that they show up in categories. —Interrobang 16:03, 24 March 2009 (EDT)

Updating the page counter

Not critical I understand, but when we initially did the migration the counter could be corrected by a script an admin could run if I recall.--RosicrucianTalk 12:15, 24 March 2009 (EDT)

References

While there's obviously no way to fully automate these, I do wonder if there's some way to automatically make them more readable.

Like being able to turn:

<sup id="cite_ref-0" class="reference">[[#cite_note-0|[1]]]</sup>

into:

<ref>1</ref>

Or turn:

<sup id="cite_ref-lorenzoiesb_16-1" class="reference">[[#cite_note-lorenzoiesb-16|[17]]]</sup>

into:

<ref name="lorenzoiesb">17</ref>

Then at the bottom, turn:

<li id="cite_note-0">[[#cite_ref-0|↑]] The movie had some content cuts to meet the standards of some <strike>oppressive regimes</strike> quaint "developing world" rating boards.

into:

0 The movie had some content cuts to meet the standards of some <strike>oppressive regimes</strike> quaint "developing world" rating boards.

And:

<li id="cite_note-tfwtour-12">↑ <sup>[[#cite_ref-tfwtour_12-0|13.0]]</sup> <sup>[[#cite_ref-tfwtour_12-1|13.1]]</sup> <sup>[[#cite_ref-tfwtour_12-2|13.2]]</sup> <sup>[[#cite_ref-tfwtour_12-3|13.3]]</sup> [http://www.tfw2005.com/boards/showthread.php?t=139273 BotCon 2007 Hasbro tour report.]

into:

tfwtour 13.0 13.1 13.2 13.3 [http://www.tfw2005.com/boards/showthread.php?t=139273 BotCon 2007 Hasbro tour report.]

And:

<li id="cite_note-murphylatino-3">[[#cite_ref-murphylatino_3-0|↑]] [http://www.latinoreview.com/news.php?id=1502 Latino Review interview with producer Don Murphy.]

into:

murphylatino [http://www.latinoreview.com/news.php?id=1502 Latino Review interview with producer Don Murphy.]

Or just something like this to turn the mess into as much usable (or at least readable) code as possible.

I don't know if this sort of processing is even possible, but if it is, it would definitely make it easier to clean up pages that have more than just a few references. --Jeysie 17:42, 24 March 2009 (EDT)

Oh, you could automate them. Same way my tool automates categories. It's just a fair amount of work. (...I suppose I should take a whack at it at SOME point.) -Derik 20:24, 25 March 2009 (EDT)
I honestly think it might be worth it... it took me literally a number of hours to untangle the references on the 2K7 movie page, and we have a fair number of similarly reference-heavy pages. (The Japanese Generation 1 Cartoon Timeline, for instance, is sitting in my bookmarks until I have the courage to tackle untangling ~60 references on a giant list. XP) Anything that can make the process even slightly easier would be well-appreciated if it's doable. --Jeysie 20:32, 25 March 2009 (EDT)

Nautilator

We have Nautilator, Nautilator (G1) and Nautilator (Energon)

  • Nautilator and Nautilator (G1) are the same guy.
  • Nautilator (G1) disambigs to Nautilator (Energon) and vice versa.
  • Nautilator is the only one of the three which has a cache version saved for it.
  • Neither Nautilator (G1) nor Nautilator (Energon) come up in web searches.

It looks to me as if Nautilator (Energon) was deleted sometime between June and the Crash and Nautilator (G1) was moved to Nautilator. Anyone know the history there? --abates 21:47, 24 March 2009 (EDT)

The Energon guy no longer exists, basically. The toy that was going to become Energon Nautilator eventually found release as a new version of the G1 guy through the fan club. So the Energon guy got absorbed back into the G1 guy's page. I think only Archerbot keeps a separate page, since he's got his own name. Maybe? I'm not sure, actually, but that's how I understand all this. -- Repowers 22:17, 24 March 2009 (EDT)
I was the guy who merged them. Archerbot wasn't an exception. —Interrobang 22:31, 24 March 2009 (EDT)
Oh! These guys! So that's why those pages didn't get updated from the cache results. --abates 23:16, 24 March 2009 (EDT)

Misplaced Pages

We have Super-God Masterforce (franchise) and Super-God Masterforce, the latter of which has more information than the former. Which one do I keep? --Jeysie 04:18, 25 March 2009 (EDT)

The latter one was resurrected with the rollback, by the looks. All the extra info is redundant with what's on Super-God Masterforce (cartoon) anyway. I suggest setting it as a redirect to the franchise page. --abates 04:54, 25 March 2009 (EDT)
That's what I figured, but I thought I'd check. OK, this issue is settled. --Jeysie 04:59, 25 March 2009 (EDT)
We've got a whole bunch of redundant pages now, ranging from Kup/Kup (G1) to Detour (Cybertron episode)/Detour (episode). Good fun! --abates 06:11, 25 March 2009 (EDT)

Transtech and TransTech are both about the canceled franchise. Shouldn't one of them be about the club comic alternate dimension? Where did that info go? --Crockalley 18:50, 27 March 2009 (EDT)

Looks like Transtech got uploaded at TransTech by accident. Both pages were in the cache zip, so I've restored them to their proper places. Thanks for pointing that out! --abates 04:37, 28 March 2009 (EDT)

Pages in Google cache but not in tfwikicache.tar.gz

So I noticed that Omnibot was suspiciously outdated, and it turns out it wasn't included in tfwikicache.tar.gz for some reason, even though it's in the Google cache, dated Mar. 9. I didn't see "Omnibot" under any of the "Lost Pages" or "Sweep" lists, so I wanted to bring it to people's attention that this phenomenon exists. I've updated Omnibot completely, but I have no idea how many other pages out there might be like this, with Google's clock still ticking on them. - Jackpot 04:41, 25 March 2009 (EDT)

That's because Omnibot, as well as various other "O" and "P" pages, are in G1MarvelBlaster's cache pile as linked under "Useful Resources" (and as mentioned several times on this page previously :>). --Jeysie 04:47, 25 March 2009 (EDT)
If no one does those by the time I finish going through all the totally slagged pages, I guess I'll be doing them. :) --abates 04:51, 25 March 2009 (EDT)
Ah, thank you. I knew there was something strange regarding the O's and P's, but I couldn't find the relevant conversation. There's.... a lot to keep up with. - Jackpot 04:56, 25 March 2009 (EDT)

Talk-page cache?

Have the talk-pages been downloaded from the Google cache en masse and put into a .zip somewhere the way the articles have? According to the stats above, only a handful have been "saved," whatever that means. - Jackpot 14:02, 25 March 2009 (EDT)

And when I try to go to any of the pages in the link marked "Saved", they don't work. Google cache just doesn't seem to think they exist at all. Yet I've seen people restore Talk pages. How is this done? - Jackpot 01:14, 28 March 2009 (EDT)
I was only able to do it when the {{bookworm}} template provided an automatic link to the page's Google cache. Now it doesn't. If I wanted to restore my Talk page, I wouldn't know how to do it.
Edit: Gah, forgot to nowiki-tag the template name.--Apcog 03:09, 28 March 2009 (EDT)
It's http://74.125.93.104/search?q=cache:tfwiki.net/wiki/(article title with underscores/encodings as needed)
So your user talkpage would be http://74.125.93.104/search?q=cache:tfwiki.net/wiki/User_talk:Apcog for instance.
Much obliged. Jackpot too, I'm sure.
Doesn't always work. Can't seem to find mine. --Lonegamer78 05:07, 28 March 2009 (EDT)
Indeed, I thank you as well. For some reason I couldn't get it to work earlier, but your cache-link is apparently better somehow. The only problem is that the cache is overwriting itself now. For instance, I was trying to get Talk:Sideways, but that's been overwritten with the March 24 version. I thought that someone had claimed to have downloaded all the Talk pages right after the crash. Am I imagining things? - Jackpot 13:52, 28 March 2009 (EDT)

Attacktix Template

I went and restored the Template:Attacktix off the Wikia copy. I know copying from there is frowned upon, but I compared it to the Google's cache of our version and made the appropriate changes. I also figured templates were a bit of a different animal as opposed to normal articles. It looks like it's ready to be used. Not knowing what our template code looked like however, I can't be certain it's exactly what it was. Are there any problems?--Tigerpaw28 02:57, 26 March 2009 (EDT)

As the original template creator, I can verify it is very much as it was when last I saw it.--RosicrucianTalk 03:07, 26 March 2009 (EDT)
We shouldn't have issues copying over templates from Wikia. All of it is our work. —Interrobang 04:05, 26 March 2009 (EDT)

Midnote template

Maybe I'm just not looking well enough, but do we still have the midnote template? If so, where is it? If not, can someone restore it, because the regular note really makes some pages look ugly? Geewunling 03:53, 26 March 2009 (EDT)

Missing images

Despite running image import TWICE, the wiki is apparently still 'missing' some images that will have to be re-uploaded by hand. (However we should actually have them all.)

When you run into missing images, please post them here so someone can tidy them;

Edgeofextinction-someonefight.jpg found on On the Edge of Extinction!
Megatron_generic_marvel_uk_265.jpg found on Once Upon a Time...
Onslaughtmodelsheetshood.jpg found on Onslaught
OptimusPrimalvsMegatron.jpg found on Optimus Primal vs Megatron!
Safeguardriseofsafeguard.jpg found on Rise of Safeguard
--Tigerpaw28 14:10, 26 March 2009 (EDT)

Image:ClubCon Raindance GrandSlam.JPG _ Raindance/Grand Slam
Image:RobotTimeMachineSixModes.jpg _ Autobot (G2)
Image:Madman comic Soundwave ejects tapes.jpg _ Decepticon Mini-Cassette
Image:Breakdown.jpg _ Breakdown (G1) --Geewunling 10:01, 27 March 2009 (EDT)

Image:SpotlightGrimlock cover2.jpg --MistaTee 13:58, 29 March 2009 (EDT)

What to do if you find a image is missing

  1. Find a cached version of the page with the image on it (at this stage, it's decreasingly likely Google will have a pre-crash version of the page, so you may be best going for the saved versions linked from the #Files section of this page).
  2. Find the image thumbnail which should be present there, and choose to "view image" or "copy image location" (in Firefox, terminology may vary in other browsers, and in IE, you may have to click "Properties" and copy the address from there).
  3. The URL will be something like http://tfwiki.net/w2/images2/thumb/b/b9/Ravageboxart.jpg/350px-Ravageboxart.jpg. Delete the sections marked in red and go to the altered link (Which should be in the form http://tfwiki.net/w2/images2/b/b9/Ravageboxart.jpg. The exact bits after /images2/ will vary, of course) to check that it works.
  4. Copy that link and go to Special:Upload. Choose to upload from a URL, and paste that link in the lower "source filename" box. Preferably fill out where it came from/etc at this point, but if you don't know, put {{bookworm|5|Needs description and copyright tag}} in the lower box and submit it (you may get a "page (but not file) exists"-type message. If so, ignore it and save the file anyway).

- SanityOrMadness 15:37, 27 March 2009 (EDT)

I think, if the image had descriptive text before, it'll show up again when the image is reuploaded. I had that happen with Image:Tww1_oracle_tank.jpg and Image:Movie_OptimusPrime_MicroIR.jpg. However, it did not happen with the image I crossed off my list above. I'm guessing that one never had a description. It might also be dependent on whether you get to the upload page by clicking on the toolbox link or by clicking the link for the missing image. I've been doing the latter.--Tigerpaw28 01:30, 28 March 2009 (EDT)
What's happened is that some of the PAGES exist, but the database has lost the FILES that are associated with them (The files are still on the server, but MediaWiki is blind to their existence). That leads to the "page (but not file) exists"-type message, and the description which existed before (which may *not* be sufficient, given the rollbacks) will be there when you go to the Image:page. - SanityOrMadness 14:40, 28 March 2009 (EDT)
I wonder if there's an easy way to search for missing images. Also, when we move servers, are we going to be able to take all of the files with us and not just the ones that the MediaWiki database knows about? --abates 16:55, 28 March 2009 (EDT)

I can't find the source for the Rise of Safeguard one. :\ And I can't find the thumbnail for Image:Stunticons-collectallfive.jpg either. Nevermind. Found them manually. Geewunling 07:23, 28 March 2009 (EDT)

I think I removed all the wikia links from the category pages. We could probably remove that bit from the main page now. There is a possibility I may have missed a few, but I did go through systematically.

I had no idea we had that many category pages. Wow. --Bluestreak7 12:02, 26 March 2009 (EDT)

Making things less tedious

Because reformatting <gallery>s is tedious, I've made a tool to do it for us.

Copy and paste the gallery code into the form, hit the submit button, copy the reformatted text back out. Will do multiple galleries, though I don't think we have many pages with more than one. --abates 20:34, 26 March 2009 (EDT)

Comic template converter

I've created a tool to convert the HTML on comic pages to {{comicstory}} and {{comicnav}} templates.

I'm pretty sure I managed to get all of the many many parameters the comicstory template can take, but please check to make sure that they match what's on the original page, and, as always, if it fails to convert something, leave a link to it on my talk page. :)

Interrobang, do you want one for the episode template? --abates 23:11, 28 March 2009 (EDT)

Wow. That's awesome! - Starfield 11:05, 29 March 2009 (EDT)

Totally scrambled pages

All right, I've fixed all of the pages that failed to convert at all and ended up as HTML mess. At least, all of the ones I could find. If anyone spots any more, feel free to drop a note on my talk page! --abates 06:56, 27 March 2009 (EDT)

Infobox template for books?

Do we have one for books like do we with comics? I can use the comic ones fine, but I can't figure out how to get the ISBN to show up in the info box and, indeed, how to the get the ISBN function that you can click to search for the book on Amazon and such retailers. Thanks. --FFN 14:57, 27 March 2009 (EDT)

The ISBN one confused me at first, but it appears if you just write an ISBN, MediaWiki recognises it and turns it into a link: ISBN 1-84023-527-6 --abates 18:22, 27 March 2009 (EDT)
We used to have a book infobox, but it must have been new enough that it didn't get saved. We need one, though, I find that trying to cram book page info into the comicstory one is just too awkward. (*would rebuild it herself, except that those infoboxes kind of fall into wikivoodoo territory code-wise*) --Jeysie 21:32, 28 March 2009 (EDT)

Redirects

Has a bot been set up to find and remove all the Wikia linkbacks? Because by doing a search for the word Wikia, I've noticed that a lot of them are on redirect pages, and that accessing the redirect page doesn't show that link until you're on the Edit page... --Detour 20:39, 27 March 2009 (EDT)

Moved/deleted pages

We got a lot of pages back which we had in June 2008 which we didn't before the crash. I did a diff on the current list of pages and the one we were saving caches from and have 43 pages left to check. So far, these pages are ones I'm not sure what happened to in the intervening 8 months:

  1. Autobot Overlord
  2. Clamp
  3. Dinocons
  4. Gone Forever
  5. Holly Kaneko
  6. Matrix Buster
  7. Monorail transport
  8. Seattle-Tacoma area
  9. Steamer
  10. Taitem Kusunoki
  11. Transformers books
  12. Transformers in 3D issue 2

Dinocons, I'd have assumed was deleted, but our cache copy of Dinobot (disambiguation) links to it. Holly Kaneko and Taitem Kusunoki are copies of Harii Kaneko and Taiten Kusunoki respectively. Someone with a better grasp of Japanese may know where they belong. Matrix buster and Seattle-Tacoma area may have been merged with Big Cannon and Seattle. Steamer, I have no idea what happened to, as Steamer (G1) redirects to Chuffer... --abates 00:50, 28 March 2009 (EDT)

Yep, I'd moved "Seattle-Tacoma area" to "Seattle" when I realized there were more specific references to that one city in the canon. - Jackpot 01:19, 28 March 2009 (EDT)
Dinocons is definitely an unrecovered page, possibly one of G1MarvelBlaster's cache recoveries... It's the Shattered Glass Dinobots who ally themselves with Decepticons in that continuity. --Detour 01:21, 28 March 2009 (EDT)
Those would be at Dinobot (Shattered Glass) though! G1MarvelBlaster's pages all start with O and P. --abates 01:42, 28 March 2009 (EDT)
Well, no. The "Dinobots" that ended up in Decepticon hands following Dungeons & Dinobots got renamed Dinocons once Do Over started. (literally. The first line of Do Over is "Good morning, Dinocons!".) --Detour 07:00, 28 March 2009 (EDT)

Transformers in 3D issue 2 was moved to Transformers in 3-D issue 2, following the title on the cover. —Interrobang 13:04, 28 March 2009 (EDT)

Double redirects

The restoration process has resulted in quite a lot of double redirects, and we probably haven't seen the end of it. I've got other projects I want to use my time on right now, but I figured I'd point this out so that anyone interested could tackle it and keep an eye on it in the future. - Jackpot 14:59, 28 March 2009 (EDT)

IDW Gen1 issues completed

These have all been fixed. Still outstanding (as far as I know) are Hearts of Steel, Beast Wars and Movie continuity. On, and reprint series. --MistaTee 11:18, 30 March 2009 (EDT)

Current status of the wiki

So, at the moment how are we? I have read all of this page but I'm still not sure what exactly is left to do at the moment (IE how many pages have we lost, how close are we to being back to normal, what is the extent of the damage to pages etc etc) P.S. how much space does the wiki take up (I'd prefer to have images and articles seperately) because I thought that maybe toghether we could all back up the wiki on our computers (as in we each get a little bit of it). P.P.S. I'll probably be registering soon, any ideas on a good name? 81.108.237.26 11:25, 30 March 2009 (EDT)

Retrieved from "https://tfwiki.duckdns.org/index.php?title=MediaWiki_talk:Community_Portal/Damage_Control_Central&oldid=252104"