any way to download the forum archives?

Give us your opinions on the new forums here.

Moderators: Peter Thomsen, miker

waterfall
Posts: 31
Joined: Sat Aug 23, 2008 6:10 am

Post by waterfall » Fri Nov 26, 2010 4:04 pm

Is there an acceptable, approved way of downloading all the topics - using a web site downloading program? Are the forum archives available for download in any other form than just going from page to page? Just asking in the interest of saving time. If there is a specific directory where to download the posts, great, but I would not want to try to download the whole site without permission. Some robots.txt files prohibit that or exclude certain directories and many forums don't encourage mass downloading of their sites. Thanks for any comments.
G.P.


User avatar
miker
Posts: 5993
Joined: Fri Mar 13, 2009 4:28 pm
Finale Version: Finale 27.4
Operating System: Mac

Post by miker » Fri Nov 26, 2010 4:10 pm

I don't know. Why not contact the forum admin? Use the CONTACT link at the upper right.
Finale 27 | SmartScorePro 64
Mac OS 13.2.1 Ventura
Copyist for Barbershop Harmony Society

User avatar
michelp
Posts: 2045
Joined: Fri Jul 25, 2003 3:35 pm
Finale Version: 27.4.1,26.3.1, Mont.
Operating System: Mac

Post by michelp » Fri Nov 26, 2010 9:55 pm

I know that the "search" function on this forum is quite limited.
But you can also use the Google search applied just to this site, and you would be surprised by the amount of results found.
In case you don't know this trick :
In the URL bar of your browser, copy/paste this :
javascript:Qr=prompt('Search%20Site%20for','');if(Qr)location.href='http://www.google.com/search?&q=site:'+ ... escape(Qr)

Then save it as a bookmark, named for instance Google this site.

When you call the bookmark, a window pops up in which you can enter a search string. The search is limited to the current site.

NB : this bookmark works with any site.
Michel
MacOsX 12.7.4, Finale 27.4.1 & 26.3.1, Mac Mini Intel Dual Core i7 3Ghz, 16 Go Ram. Azerty kb. MOTU Midi Express XT USB, Roland Sound Canvas SC-88vl, MOTU Audio Express. 2 monitors (27"' pivot, 24'"), JW Lua, RGP Lua

waterfall
Posts: 31
Joined: Sat Aug 23, 2008 6:10 am

Post by waterfall » Sat Nov 27, 2010 1:32 pm

Thanks very much michelp for the helpful post. It's useful but I'm truly interested in downloading the entire forum archives and putting them on a disc for reference in case ever needed. If I have to save the pages one at a time, then I'll do it over a period of time.
G.P.

User avatar
michelp
Posts: 2045
Joined: Fri Jul 25, 2003 3:35 pm
Finale Version: 27.4.1,26.3.1, Mont.
Operating System: Mac

Post by michelp » Sat Nov 27, 2010 1:44 pm

I perfectly understood that you want something else.
Just an idea : there are some softwares which download whole web sites for offline viewing (Sitesucker on the Mac, etc...). Maybe one of them could help...
Not sure they work for forums, database sites...
Michel
MacOsX 12.7.4, Finale 27.4.1 & 26.3.1, Mac Mini Intel Dual Core i7 3Ghz, 16 Go Ram. Azerty kb. MOTU Midi Express XT USB, Roland Sound Canvas SC-88vl, MOTU Audio Express. 2 monitors (27"' pivot, 24'"), JW Lua, RGP Lua

waterfall
Posts: 31
Joined: Sat Aug 23, 2008 6:10 am

Post by waterfall » Sat Nov 27, 2010 1:59 pm

Thanks. I've got the software on both mac and pc but I contacted the forum admin per miker's advice. Best to ask these things. Some web server will block ip addresses when there is too much and too fast download activity. So, better to find out exactly where the forum posts are and if the bulk downloading of them is approved.
G.P.

User avatar
Erik Van Geit
Posts: 32
Joined: Fri May 14, 2010 9:27 pm

Post by Erik Van Geit » Mon Jan 03, 2011 2:48 pm

Hi,

Are there any results about this question?

kind regards,
Erik

waterfall
Posts: 31
Joined: Sat Aug 23, 2008 6:10 am

Post by waterfall » Wed Jan 12, 2011 4:14 pm

forum admin emailed me with a "no" to the possibility of a bulk download
I had tried it earlier with a web site downloader and the forum is setup with the topic pages as .php files
they wouldn't open normally in my browser and the downloader couldn't save them in a form that would allow the forum to be read from disk as if I were logged in and reading the pages
it would be a great service to be able to download the forums as zip files or some other archive form but a lot of work to put them in an easily readable form.
many forums are setup to discourage bulk download and the strain that would put on their servers.
G.P.

User avatar
michelp
Posts: 2045
Joined: Fri Jul 25, 2003 3:35 pm
Finale Version: 27.4.1,26.3.1, Mont.
Operating System: Mac

Post by michelp » Wed Jan 12, 2011 4:31 pm

One thing is to downlooad the whole database, another thing is to be able to search through all the information with efficiency.
Hence my post about a better search method (see the 3rd post in this thread), which could be a reasonable way to obtain information without having to store it.
Michel
MacOsX 12.7.4, Finale 27.4.1 & 26.3.1, Mac Mini Intel Dual Core i7 3Ghz, 16 Go Ram. Azerty kb. MOTU Midi Express XT USB, Roland Sound Canvas SC-88vl, MOTU Audio Express. 2 monitors (27"' pivot, 24'"), JW Lua, RGP Lua

Post Reply