[#] Graveyard
01:25am UTC - 6/10/2011

I've started graveyarding. See LIST OF GRAVEYARDED THREADS. Completed: June 2010, July 2010, August 2010, September 2010, October 2010, November 2010, December 2010, January 2011, February 2011, March 2011, April 2011

I've also found a VPS provider and set up a box, so it should actually be a pretty simple process now to just move shit over. I'm going to finish the graveyarding within the week, move files, and have everything done by June 20. The old server will remain online until the last minute on July 1. After that, at the new host, I'll begin offering downloads for the entire archive piecemeal (e.g. August-September 2010 in a single file, torrents probable).

As before, if you see a thread you think should be preserved, name the thread number and make a case for it. Nine times out of ten, instead of it being preserved in the archive, you should rip the content out and throw it on 1d4chan. And there's always... uh, hm. It looks like Easymodo took down its /tg/ archiving service. Whoops! I guess it really is all up to me now.

Current Stats: 842 threads removed (6.3 GB), plus 57 drawthreads removed (3.8GB)

~Lord Licorice

[#] Culling and Server Move
04:08pm UTC - 5/30/2011

The server move finally has a hard deadline: July 1, 2011. I have to move sup/tg/ to a new provider by then or everything simply disappears into the ether. So that means it's time for a culling!

Like the last time, the archive is going to be purged of unwanted threads by whatever poor assholes I can rope into helping me a team of qualified, highly-trained content experts. What are the criteria, you ask? Wait, you didn't ask? WELL FUCK YOU I'M TELLING YOU ANYWAY:

Like before, the deleted threads will be moved to a graveyard where anyone who really wants that aforementioned horrible thread can still download it. (Image threads will not be given this treatment this time around. You knew better when you added it.) I will update this post or a future post with the growing list of graveyarded threads and links to download them.

When the culling is done, I will begin uploading threads by month to Dropbox, Megaupload, or whatever other service I can find as a failsafe. If I get hit by a bus, I don't want the archive to just disappear and lose the last four years' worth of amazing /tg/. I'll likely be moving to Varsbox unless someone else can suggest a good VPS provider that will fit the following criteria:

I'm going to get started on getting people to cull the archives. For those of you who give no shits either way, I've updated the archive page so that you can hide quest and draw threads now, so there's that.

Edit: For those of you helping me, I'd like you to go through a full month at a time (helped now by the handy dandy new archive page), and give me a list like the following:

Thread Number - Title - Score - Reason

You don't have to get fancy with the reason, just "quest thread" or something so I know why it should be removed. Remember, these are just recommendations; I may end up keeping a thread after all, though most of the time I'm sure the thread deserves to be eliminated. Please also include a separate list of just the thread numbers so I can plug them more easily into my tools. Extra lists of questionable items or threads to make into collections are also good.

Edit 2: Of the 6805 threads currently archived, 465 threads contain the tag 'draw' (drawfag, drawthread) and 2090 contain the tag 'quest'. That's right, a full 30% of the archive is composed of quest threads. ಠ_ಠ

Edit 3: 392 draw threads at a total of 3.8GB removed on the first pass. (Culled List) Also I guess I should clarify: I'm going to leave the "deleted" threads online until Friday or something, then they're permanently being deleted. The graveyard I'm going to leave for as long as I still have this server. I could leave Deleted and Graveyard together I guess but this gives me a directory to dump so I can get more disk space as I archive and transfer batches of threads.

Edit 4: Jesus Christ there were fucking 335 threads in May alone, 137 of which have "quest" in the tags. I need to look about changing the way threads get archived, or be extremely aggressive in future culls. Maybe something like deleting threads below 5 or 10 or something, or (ugh) requiring multiple votes to add a thread ala 4chanarchive.

Ultimate question: Should I ban quest threads, or just aggressively cull them?

Edit 5: Anon raised a good point that made me wonder: how long does Easymodo store threads? Do they have a purging policy or any of that? How far back does it go? Hell, it'd make my life massively easier if I just hosted a listing service for /tg/ that linked to archived threads on Easymodo. Would Eksopl mind? What does he use for a hosting provider, anyway? QUESTIONS, MAN.

If anyone could put me in touch with Eksopl so I can ask some questions about how he runs things over there, I'd be much obliged. Either I could just set the archive to link to Easymodo, or buy hosting from the same company.

~Lord Licorice


[#] 4chan issues
02:05am UTC - 4/12/2011
Hey guys, 4chan's been having some load issues for the past couple of days, which is leading to weird problems (header/footer doubling, pages dying halfway through, etc.). I've added a sanity check to make it fail gracefully if the page doesn't load all the way, but pages may not update properly until 4chan sorts the problems out.

Fortunately, I had the foresight to make the archive save multiple page copies (first request, second-to-last request, and last request). I'll fix the broken threads, usually by replacing them with the older version. It's not perfect but it's better than losing the thread entirely. Speaking of which, the following are threads I need to fix manually: 14562610, 14556877, 14560920, 14542831, 14545566, 14550655, 14551718, 14536281. If you find other broken threads, please comment.

~Lord Licorice


[#] Happy April 1! Again!
07:20am UTC - 4/01/2011

Happy April Fool's Day everyone! I hope you're enjoying the shit all over the internet today! It's also fucking 7:30 in the morning christ I need sleep oh god why am I awake.

Only three pages today, I'll likely add one or two more later, but sleep now, oh god why. Old pages:

~Lord Licorice


[#] Happy New Year
08:40pm UTC - 12/31/2010
Happy New Year, motherfuckerrrsssss

~Lord Licorice

[#] Happy Turkey Massacre Day!
10:25pm UTC - 11/25/2010

~Lord Licorice

[#] Goddamnit.
07:54am UTC - 10/31/2010
So because my life is a fine example of decay and entropy in action, the PSU on the old sup/tg/ server died today. This is the model I need to replace; if anyone can find this cheaper and/or with a more reliable vendor than random people on Ebay, it'd be much appreciated. (Likewise, any leads on cheap, legit copies of Win7 Professional or better would be great.)

Also, the Minecraft server is up again. The map's been reset; new rules are (to start) 20x20 plot for your house in the sup/tg/ town, no ostentatious displays (lavafalls, etc.), no super tall structures, no penis towers. We don't have mobs yet, but the idea is to build a community to fend them off, rather than a bunch of free-standing art installations. Check the IRC for details. BOTHER PURPLE ABOUT PLUGINS NOT WORKING

~Lord Licorice

02:15am UTC - 10/31/2010



~Lord Licorice


[#] Oktoberfeeeeeeeeeest
05:12am UTC - 10/01/2010
oh hai /tg/

It's been... a while. An update on the archive, and clarification: The threads that need to be moved have been moved, and everything is in its place. (Yes, that misnamed quest thread was moved and renamed properly.) The old site and graveyard will remain online indefinitely, so a lot of the "I don't know if we'll keep this" content will just be there if anyone needs it. Similarly, the art and drawthreads and such are all still there, so if someone would like to (gently!) pull these and store them somewhere, it'd be appreciated. (No one's volunteered yet, despite all the clamoring requests to preserve it... with one exception but I don't know what happened to him or the content.)

I haven't updated the culling tools to work on the new hosting yet, so I haven't culled June, July, or August. The graveyard policy for the new hosting will go back to normal (deletion during the next culling). Provided I don't install any new gameservers (or just remove others to make room) I've got ample breathing room here, so I will continue to host sup/tg/ and the archives for the foreseeable future (provided I don't end up dead or something).

Let me know if I missed something, something's broken, etc. etc. etc.

~Lord Licorice


[#] Almost done!
12:53am UTC - 8/26/2010
The threads are completely moved over with the exception of the threads I was supposed to resurrect (going to do that manually and move them later), and I'm currently working on improving the viewing page. I might even work on culling July if I'm feeling particularly ambitious.

Other than those last few threads to save, everything's back up and running. I'm just going to leave the old server up and running, if anyone needs or wants anything they're free to grab it whenever. Hooraaay

Edit: By the way, any votes you've made on old threads have been lost. You shouldn't have been voting on them anyway, but, you know, just letting you know. Any votes after July 10 were kept. Similarly, for those of you who were still archiving threads on the old site somehow and submitting votes there are a particular brand of derp.

~Lord Licorice

[#] Slow Archive Progress
06:26am UTC - 8/09/2010
Hey all, quick update. Sorry about the glacial pace of the restoration; I'm trying not to fuck it up when I move it, and I've been having trouble properly getting these threads to compress (and then extract with full paths to the right location, which is, you know, kinda important, what the fuck 7z why is that the common command).

I've also been fucking with server things in general, such as the brand new Minecraft server; let's just say that Notch really needs to hop in a time machine and write the game in anything other than fucking Java. Goddamned memory leaks cause Java to die, and all content in memory is lost, which seems to be all changes since the last restart. ffs. It now reboots every 4 hours starting at midnight UTC so that should fix it.

Anyway, as we speak, the first batch of threads are decompressing on the server (in the right directories, with the right file permissions); I've been just going by thread number and moving newest-oldest. I'm restoring back through thread 10011240, which is around May 24, 2010. (Edit: It worked! Huzzah!)

Once this proves to have worked just fine and I can stop worrying, I'll do another batch tonight, and so on. I've got a few days of vacation coming up this weekend, so I'm going to focus and get this shit done. I apologize for all the delays, but it's not like the content's going anywhere. Besides, the extra delay has given people way, way more than enough time to salvage whatever fapfics they'd like to save before I decommission the old server; I'll make a formal announcement this weekend when it's going dark (and even then I'll probably still leave it up for a while. I'm nice like that).

Update: Been pluggin' along. Somewhere in the 7zillions now. Uploading ~2GB a night or so, should be done within the coming week.

Update 2: I've lost a couple days due to Comcast asshattery, since I couldn't upload a 2.xGB file during constant spikes/drops and huge lag. See, before, when the internet would drop or slow to a crawl, I'd assume it was due to load on sup/tg/; now I'm finding it's just as likely Comcast's fault.

Also I decided I'm probably never taking down the old server. The bandwidth consumption dropped to nearly nothing (I figured it'd still get a bit hammered now and then) and I'm way, way under my bandwidth cap, so the graveyard on the old server will remain online indefinitely. (The new graveyard, e.g. from July 2010 on, will be deleted regularly.)

~Lord Licorice

[#] Culling complete!
04:10am UTC - 7/30/2010

We're done culling the archives! Yaaay!

You presently have 48 hours* until Friday, August 6* to get everything you want from the following list:

Culled List

*or however long it takes me to turn off the server.

Anyway, the links on that page have all been updated to the old website, so you can download whatever you'd like to keep. Do not scrape the website or you will be banned. Art dumps and drawfag threads will be uploaded later to MegaUpload or something so don't bother with those, you'll get them in a tidy .rar at some point, or they're already on tgchan (I'll get the links for those and add them here). If anyone would like to officially or unofficially start transcribing writefag threads to 1d4chan please go ahead.

Remember, this is a general 48-hour window, but I'm not going to delete the threads for a long while (if ever), so if you miss it don't fret. Just ask and I'll be happy to send you a copy of whatever you'd like. Again, don't scrape the site because you feel like everything deserves to be saved; it really, really fucking doesn't, and if you haven't seen the culled list yet, you'll quickly figure out why.

After the window I'll start moving ~5GB of the surviving threads to the Xen a day until it's up to speed, cull July, and we should be all set. By next week or so we should be up to speed, unless of course I get sucked into Minecraft to the exclusion of all else... again...

Edity: Oh yeah, you'll notice the "saved" thread still show up in the culled list. I can't be fucked to bother removing them from the list, but if you asked me to save something (like those mislabeled quest threads and the like) I'll be restoring them during the move.

Edity 2: I've decided to leave the graveyard up until at least Friday, if not beyond that; the bandwidth usage from people snagging "killed" threads is way less than I thought it was going to be. Remember, the graveyard is here. I'm going to start transferring tonight.

~Lord Licorice

[#] Archive update
05:23am UTC - 7/27/2010

Everything's proceeding quite a bit slower than I'd planned, but it's still proceeding. I'm almost ready to begin uploading; I figure I'll move things in an orderly fashion, something like 5GB a day, until the archives are back up to normal. As you've likely noticed, the request interface and new threads still work, so it's not like we've been losing new content during this downtime. (And Comcast hasn't exactly been stable lately, so I'm going to pick a good time and start uploading.)

As before, add any requests to save material to the post below this one (do not ask for threads to be saved anywhere else or they may be missed!).

~Lord Licorice

[#] Archive culling battle plans
10:11pm UTC - 7/12/2010

Okay, so here's the deal. I've roped some poor suckers politely asked a few members of the sup/tg/ IRC to help me cull the horrible, horrible threads in the archive so whittle the filesize down to a manageable level. Their guidelines are:

  1. Kill all drawthreads. These aren't any different than regular image dump threads, except they're singularly lower in quality and purpose; they're 10% content, 90% begging the artist. Any good drawfags will have their own dA account or at least a page on 1d4chan where their gallery is set up. Anyone participating in the threads likely saved "their" images, so why archive it?
  2. Kill failed and low-quality quests. There are a metric assload (that's a real measurement, look it up) of terrible threads in the archive. Many of them are "part one" to a one-part series or generally terrible. They're going away. Huge quests (regardless of quality...) are being tossed into the Collections section.
  3. Kill threads already against the rules. This means every image dump, every rapefic, every RP thread is being culled, because they shouldn't be there anyway.
  4. Kill troll, shitthreads, and general poor content. People have a tendency to archive a thread if they smirked slightly at one post amongst a hundred shitposts. We're rectifying this.

As mentioned previously, the list of culled threads is here. You'll notice more subentries for individual months, and the pastebins where the aforementioned chumps beautiful, beautiful people handed me their lists, along with their commentary or reasoning. If you've got a problem with something being removed, well, first, think long and fucking hard before you ask, and if you still decide it's really worthy of remaining, let me know in the IRC.

Once we're done culling, which may take a couple days (the archives are still accepting new threads), I will restore everything at the old location for 48 hours. This will give everyone enough time to retrieve content from the graveyard (surviving threads will probably not be restored yet at that stage). After that, I'm going to completely wipe the graveyard, upload the content back in stages, and finally decommission the old server for good.

Special thanks to schlubs goddamned Internet Heroes Aeros, PurpleXVI, Phaethon and Symmetry for mucking out the e-stables for me. Five months' worth of threads, anything under -5 and all threads tagged 'image dump' have dropped 8.75GB, and we've got many months to go.

Edit: If anyone has an answer for where to put the drawfag content, please let me know. I can't keep it for bandwidth/space reasons and certain elements inform me that 1d4chan isn't the place for them (apparently it's not the place for fucking anything). I don't want to start administering a drawfag gallery, which is why 1d4chan would have been ideal, so if someone has a solid suggestion on where I can upload everything just so people can get copies of it or something, let me know in the comments.

Edit 2: 2010 is done and most of 2009; only three months graveyarded so far but we're up to 10.3GB culled! Yaaay progress.

Edit 3: 2009 is done, total of 15.4GB removed. 2008 to go and then we start the migration.

Edit 4: 2008 is done, up to 17.5GB removed. Going to hit 2007 and start uploading piecemeal.

~Lord Licorice

[#] Moving the site
06:33pm UTC - 7/11/2010

Important notice!

So it looks like Comcast has finally started actually enforcing their bandwidth caps, which are presently 250GB per month; I've used 450-475GB for the last three, and it looks like at least 300+ of that has been from the site. While I'd normally be kinda proud about that, they've threatened to terminate my service if I don't cut it out...

Over the next couple days, I'm going to be transferring sup/tg/ to the Xen server. This does have a couple benefits, such as being able to throttle bandwidth limits (to prevent scraping) and provide a faster site experience in general. The downside, of course, is that this is going to take me a lot of time and effort to move everything and get it running the way it was before.

To help make this easier on myself, I'm also going to be doing a vicious purge of the archives, since there's just a lot of shit I've left there and figured "eh, why not?" Now I need to transfer 40GB of excrement flecked with nuggets of gold over to the new location, so it's time I started mucking out the stables a bit.

It's going to be pretty bumpy, but I don't foresee any major issues, other than the usual weird complaints about culling and a few things that ran in a Windows environment that I'll need to reconfigure under Debian (like the PDF creator for the Dark Heresy chargen). Who knows, maybe all this newness will help kick me into actually completing any of those projects I've been putting off, like updating the DH chargen or creating an EP chargen.

The current list of graveyarded threads is here, though you won't be able to view them yet. As I said, this is a very strict culling, so I may kill an item that deserves saving (particularly pieces of quest threads that may or may not need to be kept for completion purposes). Point out anything I miss in the comments here or in the IRC.

As noted, many parts of the site won't function until this is done, and a few pieces are going away entirely (the art archives for example). Don't bother reporting things unless they're still screwed up a week from now.

Addendum: I've put people to work slaving in the horrormines to start culling shit. Be prepared for lengthy lists. I've also moved Maid Quest and Frost Giantess Quest under the Collections section (which may not show up until I'm done moving shit around).

~Lord Licorice

Go Older - Go Newer


Site code and contents © 2007-2017 All rights reserved. Click here for legal information.
If you are under the age of 18, please leave this site immediately. Asshole.