Awesome news, Kreisher! Just got my PMs downloaded.
Gonna try your command later, grinninglich, as I am probably the only other one in here running a flavor of Linux. You saved me from having to dig through wget’s man page
Im done downloading it. When i left i saw 7+gb and it was still going, i got home it said that it was finished but now i see that the whole thing is 2gb. How can that be? I got a few errors in the log too.
Any idea which is the default save folder of that program?
I’m at 3 GiB by now and still going. Another 4 GiB is probably going to take a few more hours on my connection… I’m using a different program however, so maybe the outcome will be different as well.
I dont even know if what i downloaded was the whole thing. Before i go out i was seeing almost 8gb, a few hours later i get home and it says “finished” and in “my websites” folder the whole TQ thing is 2gb only. Although i did browse and it does work. Now im just not sure if its the whole forum or what. Also its pretty slow to load the pages…
You are probably correct about about the post attachements. Though I thought you had to be logged into the site to download attachments.
I have mine set to dl external attachments, but I’m pretty sure only images apply here. I doubt it’s getting any external mod files as these aren’t immediately available via a link. Most Dropbox, mega, etc. attachments require user action before the dl starts.
Currently sitting at 16.5gb downloaded with 17218 links scanned and 144801 links detected.
Has anyone who was making a mirror managed to get a complete one of the site yet? My first attempt had to be restarted and now I am at 13.98 GB with 307 errors and 50k links scanned out of roughly 220k possible links. Just wanted to see what we are looking at here for a final total.
@Pazuzu - I’d check your log files. I’d be shocked if you have the whole site.
I have been reading this thread, found by accident after googling for some informations regarding TQ, since the shutdown of Titan Quest Forum and just registered today. I may not have been one of the greatest poster in the old forum but when it went down I felt like loosing a old friend and futhermore that forum was like a never ending encyclopedia of knowledge and information.
Even if it is back on line, hoping for a long time, I would like to say a great thank you to all the people that work so hard to find a solution to get the old forum back.
Sorry, but I missed the why in the project of many people simultaneously mirroring the site.
Also curious what you get when the mirror is done, a bunch of static html pages or the components needed to reinstate a vbulletin site? Evidently, vbulletn costs a lot of money and it sucks balls. What you should be after is whatever kinda migration data you need to recreate the website with free forum software. Then you can reinstate the forums in a functional manner, albeit with a different look and feel likely depending on the choice of forum software.
I’m a lot further along, but unfortunately I’m getting the same speed. The HTTracker engine hasn’t been updated in years and doesn’t support “modern” computing capabilities. There are theoretically ways to force multi-thread support using command line, but I wasn’t able to find anyone who was able to make it work. There are a few tricks you can do that drastically improve the odds it won’t give up early though.
First: Increase the number of links before it gives up from 100k to 1mil
Second: Set it retry attempts to at least 2.
Third: Increase the allowed active connections to 8.
Lastly: Follow the instructions on this video. I don’t know why these tweaks work, but they’ve improved my trackers reliability dramatically.
If / when mine completes, I plan to package it up and share it with everyone.