IMDB archive project - where is it?

I don't know where the Archive Team keeps the data, but read this:

https://www.themoviedb.org/talk/58aaa9f192514141a0000d7e?language=en

Someone made a torrent for an archive that contains 1.9 million posts. And people who downloaded it say it's legit. (if you believe them).

That's all the info I have about archives.

According to to a reply on their Github project page, the files will eventually end up here:

https://archive.org/details/archiveteam_imdb

And people could probably ask more questions over at this Github project issue:

https://github.com/ArchiveTeam/imdb-grab/issues/1

@Apis, I've downloaded that torrent, and it is indeed legitimate. Not only that, but it's impressive. It doesn't feature any off the actor boards, and doesn't seem to have any of the television show boards either. But it has a gigantic collection of movie boards. Every movie board I've searched for within it is there, even some very obscure ones. Not only that, but the boards, once extracted from the archive, are fully browsable, just like a real website. And each thread is preserved as one long page, with every post from every page of the original thread put together on one seamless page. I've never seen anything like it before.

@jim, if you're reading this, I highly recommend that you check out the torrent. It would help you fill many of the gaps in your current archive of IMDb posts, including much of the 9-reply limit problem that currently exists.

@Alexander the Drake, I believe that the IMDb boards that Archive Team download are already uploaded to Archive.org at the link you posted. They are apparently contained in multiple huge .gz archive files that can be downloaded and extracted. My guess is that it's probably the most complete of the IMDb boards out there, but the files are too huge for me to download (just downloading one of the archive files alone would use 1/3 of the data allowance that my ISP gives me).

It would be great if all these projects could be merged into one giant online archive, which then could be the basis for a new website like IMDb.

@hbenthow

I think a bunch of the Archive Team's stuff is still not up. The most recent date on the files available is the 17th, if I'm reading the timestamps correctly, and the distributed scraping project was still running until the unexpectedly early shutdown.

According to the tracker, they managed to grab 3973 GB's worth of 7502012 items, where I think an item was an entire name/title board's contents (as far as I could tell from the messages scrolling on the console while I was running my copy):

http://tracker.archiveteam.org/imdb/

So hopefully they managed to pick up the stuff that other places are missing.

Anyway, glad to hear that the TMDB-posted torrent is legit and seems to have even the really obscure stuff, movie-wise. I was saving a bunch of the title boards for some foreign language films I had on my watching wishlist but only managed to grab maybe a dozen or so between doing it by hand for the smaller ones (Scrapbook X extension on Firefox with the auto-save add-on is very useful for this sort of thing) and using the script methods you posted in your thread about archiving (many thanks for that, BTW!).

It's nice to know that there's a good likelihood of the other boards for old Hollywood stuff that were further down on my priority save list are probably there to read once I finally get around to watching those.

@Stratego

ISTR I read somewhere (sorry, can't remember where or I'd link it) that the end-goal for the Archive Team was to incorporate their files into a browseable version on the Wayback Machine or some other similar dedicated project on Archive.org.

@ hbenthow

I'm downloading it right now. For sentimental reasons, if nothing else. :)

@Alexander the Drake

I truly hope that's going to happen, because even though it's possible to start all over, those old threads are invaluable. Every time I watched a movie I would check out the boards to read the discussions, even if I didn't contribute to them myself most of the time. I REALLY need something similar back in my life again!

-