(oh god has it really been that long since my last update, jesus)
We had a few hours' outage today as the VPS provider had to reboot the cluster containing sup/tg/ and perform some work and monitoring on it. It's a little shaky right now, but otherwise things seem to be back to (ab)normal. I'll keep an eye on it, and don't expect at this point to see any other issues in the near future.
The old sup/tg/ server has been officially powered down. I'll definitely miss it, but users likely won't even notice the difference. One plus is the plethora of space, which has allowed me to copy over the graveyard and drawthreads as well; I thought they'd be deleted forever, but with so much extra room, I figured it'd be safer to keep on-hand. I apologize to anyone that panicked about favorite threads being lost, but I have them in massive 7zip files if I need to recover something.
Suddenly, Muju Little Ponythulhu: Ftaghn is Magic:
Congratulations! If you're seeing this, you're at the brand new site. I'll add more notes here as I continue to migrate, but for now, you're at the right place. Note: IRC is at irc.thisisnotatrueending.com; if you were using xen.thisisnotatrueending.com, it'll time out, so check your settings.
The IRC and website migration went off without a hitch, which is a pleasant surprise given my usual track record with this sort of thing. As far as I'm presently aware, the entire archive should be moved and available, so you can start browsing, voting and adding new threads. If you run into any weird issues, please post them here, along with the URL and full error message. (Yes, the game, player and hamachi listings are all down right now, but I need to clean out the spam that got into them first...)
Reminder: July 1 is when the old server goes down permanently, including the graveyard. LIST OF GRAVEYARDED THREADS If there's a good thread you want saved, make a case for it in the comments here, otherwise just save a copy (I suggest DownThemAll!) or transcribe the content onto 1d4chan.
Once I'm satisfied that everything is working correctly, I'll be hosting torrents and direct downloads of the entire archive, from October 2007 through June 2011. I already have the files, I just need to make them available without taxing the server too heavily. Fortunately, I've got more hard drive space and bandwidth than I know what to do with now; I don't know what the pageload speed will be like under the new provider, but you're welcome to let me know if it's faster/slower, I'm curious.
And now, as it's 6am, I'm going to sleep. There's a couple other services to migrate (TeamSpeak, icecast, etc.), but it can certainly wait. Let me know if anything breaks.
Edit June 22: I just found a config error that prevented threads archived in the past 24 hours from being correctly updated; you may see some threads with incomplete content. If you happen to have the full HTML for these threads (likely quest authors) let me know and I'll update the thread with your copy.
Edit June 29: Per popular request, I've resurrected LoliQuest from the graveyard (and even grabbed the very first entry off Easymodo, which apparently put the /tg/ archives back up). I'm adding them back to the database and uploading the files as I type this. For anyone that wants the graveyard 7zip, PM me on the IRC and I will give you the download link for the file. The graveyard and old server are going away on July 1 or shortly thereafter, so you've got two days to get it if you want it!
I've started graveyarding. See LIST OF GRAVEYARDED THREADS. Completed: June 2010, July 2010, August 2010, September 2010, October 2010, November 2010, December 2010, January 2011, February 2011, March 2011, April 2011
I've also found a VPS provider and set up a box, so it should actually be a pretty simple process now to just move shit over. I'm going to finish the graveyarding within the week, move files, and have everything done by June 20. The old server will remain online until the last minute on July 1. After that, at the new host, I'll begin offering downloads for the entire archive piecemeal (e.g. August-September 2010 in a single file, torrents probable).
As before, if you see a thread you think should be preserved, name the thread number and make a case for it. Nine times out of ten, instead of it being preserved in the archive, you should rip the content out and throw it on 1d4chan. And there's always... uh, hm. It looks like Easymodo took down its /tg/ archiving service. Whoops! I guess it really is all up to me now.
Current Stats: 842 threads removed (6.3 GB), plus 57 drawthreads removed (3.8GB)
Culling and Server Move
04:08pm UTC - 5/30/2011
The server move finally has a hard deadline: July 1, 2011. I have to move sup/tg/ to a new provider by then or everything simply disappears into the ether. So that means it's time for a culling!
Like the last time, the archive is going to be purged of unwanted threads by
whatever poor assholes I can rope into helping me a team of qualified, highly-trained content experts. What are the criteria, you ask? Wait, you didn't ask? WELL FUCK YOU I'M TELLING YOU ANYWAY:
- Image dumps: These were never allowed to begin with, and comprise a sizale chunk of the archive's 40GB.
- Drawthreads: These are image dumps, except vaguely sanctioned; as I indicated many moons ago, you can archive these as long as you acknowledge and expect they will be eventually deleted. That time is now.
- Shit threads: Threads under a certain score with little to no activity or apparent merit will be removed. This means deeply negative scores, rapefic threads, LOL EPIC HIVEMIND, Part 1 of 1 quests with no views, and so on.
- Bad quest threads: Quests that are objectively stupid, low postcount, low creativity, low votes, Part 1 of 1.
- Anything that conflicts with the rules listed on the request interface. High-scoring threads will receive special consideration but if there's a 50+ scoring graphic description of eldar elfgirl catboy incest, it's getting tossed.
Like before, the deleted threads will be moved to a graveyard where anyone who really wants that aforementioned horrible thread can still download it. (Image threads will not be given this treatment this time around. You knew better when you added it.) I will update this post or a future post with the growing list of graveyarded threads and links to download them.
When the culling is done, I will begin uploading threads by month to Dropbox, Megaupload, or whatever other service I can find as a failsafe. If I get hit by a bus, I don't want the archive to just disappear and lose the last four years' worth of amazing /tg/. I'll likely be moving to Varsbox unless someone else can suggest a good VPS provider that will fit the following criteria:
- 450GB+ transit a month
- 60-80GB+ HDD space
- Allows IRC, game servers, etc.
- Root access (I need more than just vanilla webhosting, I need full access to my own instance, preferably Debian)
- Not a consideration: dedicated hosting (shared is fine), memory (IRC and the website are the most I really need), bandwidth (I'd love to keep running the icecast server but it's not necessary)
I'm going to get started on getting people to cull the archives. For those of you who give no shits either way, I've updated the archive page so that you can hide quest and draw threads now, so there's that.
Edit: For those of you helping me, I'd like you to go through a full month at a time (helped now by the handy dandy new archive page), and give me a list like the following:
Thread Number - Title - Score - Reason
You don't have to get fancy with the reason, just "quest thread" or something so I know why it should be removed. Remember, these are just recommendations; I may end up keeping a thread after all, though most of the time I'm sure the thread deserves to be eliminated. Please also include a separate list of just the thread numbers so I can plug them more easily into my tools. Extra lists of questionable items or threads to make into collections are also good.
Edit 2: Of the 6805 threads currently archived, 465 threads contain the tag 'draw' (drawfag, drawthread) and 2090 contain the tag 'quest'. That's right, a full 30% of the archive is composed of quest threads. ಠ_ಠ
Edit 3: 392 draw threads at a total of 3.8GB removed on the first pass. (Culled List) Also I guess I should clarify: I'm going to leave the "deleted" threads online until Friday or something, then they're permanently being deleted. The graveyard I'm going to leave for as long as I still have this server. I could leave Deleted and Graveyard together I guess but this gives me a directory to dump so I can get more disk space as I archive and transfer batches of threads.
Edit 4: Jesus Christ there were fucking 335 threads in May alone, 137 of which have "quest" in the tags. I need to look about changing the way threads get archived, or be extremely aggressive in future culls. Maybe something like deleting threads below 5 or 10 or something, or (ugh) requiring multiple votes to add a thread ala 4chanarchive.
Ultimate question: Should I ban quest threads, or just aggressively cull them?
Edit 5: Anon raised a good point that made me wonder: how long does Easymodo store threads? Do they have a purging policy or any of that? How far back does it go? Hell, it'd make my life massively easier if I just hosted a listing service for /tg/ that linked to archived threads on Easymodo. Would Eksopl mind? What does he use for a hosting provider, anyway? QUESTIONS, MAN.
If anyone could put me in touch with Eksopl so I can ask some questions about how he runs things over there, I'd be much obliged. Either I could just set the archive to link to Easymodo, or buy hosting from the same company.
Hey guys, 4chan's been having some load issues for the past couple of days, which is leading to weird problems (header/footer doubling, pages dying halfway through, etc.). I've added a sanity check to make it fail gracefully if the page doesn't load all the way, but pages may not update properly until 4chan sorts the problems out.
Fortunately, I had the foresight to make the archive save multiple page copies (first request, second-to-last request, and last request). I'll fix the broken threads, usually by replacing them with the older version. It's not perfect but it's better than losing the thread entirely. Speaking of which, the following are threads I need to fix manually: 14562610, 14556877, 14560920, 14542831, 14545566, 14550655, 14551718, 14536281. If you find other broken threads, please comment.
Happy April 1! Again!
07:20am UTC - 4/01/2011
Happy April Fool's Day everyone! I hope you're enjoying the shit all over the internet today! It's also fucking 7:30 in the morning christ I need sleep oh god why am I awake.
Only three pages today, I'll likely add one or two more later, but sleep now, oh god why. Old pages:
Happy New Year, motherfuckerrrsssss
Happy Turkey Massacre Day!
10:25pm UTC - 11/25/2010
So because my life is a fine example of decay and entropy in action, the PSU on the old sup/tg/ server died today. This
is the model I need to replace; if anyone can find this cheaper and/or with a more reliable vendor than random people on Ebay, it'd be much appreciated. (Likewise, any leads on cheap, legit copies of Win7 Professional or better would be great.)
Also, the Minecraft server is up again. The map's been reset; new rules are (to start) 20x20 plot for your house in the sup/tg/ town, no ostentatious displays (lavafalls, etc.), no super tall structures, no penis towers
. We don't have mobs yet, but the idea is to build a community to fend them off, rather than a bunch of free-standing art installations. Check the IRC for details. BOTHER PURPLE ABOUT PLUGINS NOT WORKING
BEAR CHEF SAYS
oh hai /tg/
It's been... a while. An update on the archive, and clarification: The threads that need to be moved have been moved, and everything is in its place. (Yes, that misnamed quest thread was moved and renamed properly.) The old site and graveyard will remain online indefinitely, so a lot of the "I don't know if we'll keep this" content will just be there if anyone needs it. Similarly, the art and drawthreads and such are all still there, so if someone would like to (gently!) pull these and store them somewhere, it'd be appreciated. (No one's volunteered yet, despite all the clamoring requests to preserve it... with one exception but I don't know what happened to him or the content.)
I haven't updated the culling tools to work on the new hosting yet, so I haven't culled June, July, or August. The graveyard policy for the new hosting will go back to normal (deletion during the next culling). Provided I don't install any new gameservers (or just remove others to make room) I've got ample breathing room here, so I will continue to host sup/tg/ and the archives for the foreseeable future (provided I don't end up dead or something).
Let me know if I missed something, something's broken, etc. etc. etc.
The threads are completely moved over with the exception of the threads I was supposed to resurrect (going to do that manually and move them later), and I'm currently working on improving the viewing page. I might even work on culling July if I'm feeling particularly ambitious.
Other than those last few threads to save, everything's back up and running. I'm just going to leave the old server up and running, if anyone needs or wants anything they're free to grab it whenever. HooraaayEdit:
By the way, any votes you've made on old threads have been lost. You shouldn't have been voting on them anyway, but, you know, just letting you know. Any votes after July 10 were kept. Similarly, for those of you who were still archiving threads on the old site somehow
and submitting votes there are a particular brand of derp.
Slow Archive Progress
06:26am UTC - 8/09/2010
Hey all, quick update. Sorry about the glacial pace of the restoration; I'm trying not to fuck it up when I move it, and I've been having trouble properly getting these threads to compress (and then extract with full paths to the right location, which is, you know, kinda important
, what the fuck 7z why is that the common command).
I've also been fucking with server things in general, such as the brand new Minecraft server; let's just say that Notch really needs to hop in a time machine and write the game in anything other than fucking Java
. Goddamned memory leaks cause Java to die, and all content in memory is lost, which seems to be all changes since the last restart
. ffs. It now reboots every 4 hours starting at midnight UTC so that should fix it.
Anyway, as we speak, the first batch of threads are decompressing on the server (in the right directories, with the right file permissions); I've been just going by thread number and moving newest-oldest. I'm restoring back through thread 10011240, which is around May 24, 2010. (Edit:
It worked! Huzzah!)
Once this proves to have worked just fine and I can stop worrying, I'll do another batch tonight, and so on. I've got a few days of vacation coming up this weekend, so I'm going to focus and get this shit done. I apologize for all the delays, but it's not like the content's going anywhere. Besides, the extra delay has given people way, way more than enough time to salvage whatever fapfics they'd like to save before I decommission the old server; I'll make a formal announcement this weekend when it's going dark (and even then I'll probably still leave it up for a while. I'm nice like that).Update:
Been pluggin' along. Somewhere in the 7zillions now. Uploading ~2GB a night or so, should be done within the coming week.Update 2:
I've lost a couple days due to Comcast asshattery, since I couldn't upload a 2.xGB file during constant spikes/drops and huge lag. See, before, when the internet would drop or slow to a crawl, I'd assume it was due to load on sup/tg/; now I'm finding it's just as likely Comcast's fault.
Also I decided I'm probably never taking down the old server. The bandwidth consumption dropped to nearly nothing (I figured it'd still get a bit hammered now and then) and I'm way, way under my bandwidth cap, so the graveyard on the old server will remain online indefinitely.
(The new graveyard, e.g. from July 2010 on, will be deleted regularly.)
Go Older - Go Newer