Author Topic: The Chronology of the Blockland Forums  (Read 286710 times)

Archive.org has a fairly expansive history of the BLF
Yeah, that's true. But I want something client-side that doesn't rely on a server. Maybe something I can save on a hard drive?

Yeah, that's true. But I want something client-side that doesn't rely on a server. Maybe something I can save on a hard drive?
at the very start of 2019 i ran a custom downloader on the blf to download every blf topic (along with almost every linked image and attached files). right now its not in a browsable state however, as its just raw html files not yet localised or put in a proper (searchable!) database.
the total size of the archive is 145.1gb, though just the html files is only 25.8gb. once parsed into a database, this would reduce the filesize quite a bit i imagine

at the very start of 2019 i ran a custom downloader on the blf to download every blf topic (along with almost every linked image and attached files). right now its not in a browsable state however, as its just raw html files not yet localised or put in a proper (searchable!) database.
the total size of the archive is 145.1gb, though just the html files is only 25.8gb. once parsed into a database, this would reduce the filesize quite a bit i imagine
Holler at me when you get it into something usable, would love to have the whole forums as a keepsake.

at the very start of 2019 i ran a custom downloader on the blf to download every blf topic (along with almost every linked image and attached files). right now its not in a browsable state however, as its just raw html files not yet localised or put in a proper (searchable!) database.
the total size of the archive is 145.1gb, though just the html files is only 25.8gb. once parsed into a database, this would reduce the filesize quite a bit i imagine
Did you use Heritrix? I'm actually running an archive job right now.
« Last Edit: October 26, 2019, 11:19:10 PM by Pecon »

Did you use Heritrix? I'm actually running an archive job right now.
no, i wasnt even aware this program existed. i wrote a (probably not very good) program in nodejs utilising wget which runs through every topic id and downloads them (along with detecting for extra pages and following them too), with functions for downloading attachments and getting source photobucket images. doing all this with my login cookies to bypass the forum word filter.
i have yet to download profiles though


day discussion thread dies

should also add the night night discussion thread died as well :(

They can still be posted in, they're just not gonna be updated anymore

I think it's best to wait until they can't be posted in before declaring them dead


haha what if the forum ends at the same time as the world in 400 days haha


haha what if the forum ends at the same time as the world in 400 days haha
a fitting end to the roach that wont die