any way to download the forum archives?
Moderators: Peter Thomsen, miker
Is there an acceptable, approved way of downloading all the topics - using a web site downloading program? Are the forum archives available for download in any other form than just going from page to page? Just asking in the interest of saving time. If there is a specific directory where to download the posts, great, but I would not want to try to download the whole site without permission. Some robots.txt files prohibit that or exclude certain directories and many forums don't encourage mass downloading of their sites. Thanks for any comments.
G.P.
- michelp
- Posts: 2073
- Joined: Fri Jul 25, 2003 3:35 pm
- Finale Version: 27.4.1,26.3.1, Mont.
- Operating System: Mac
I know that the "search" function on this forum is quite limited.
But you can also use the Google search applied just to this site, and you would be surprised by the amount of results found.
In case you don't know this trick :
In the URL bar of your browser, copy/paste this :
javascript:Qr=prompt('Search%20Site%20for','');if(Qr)location.href='http://www.google.com/search?&q=site:'+ ... escape(Qr)
Then save it as a bookmark, named for instance Google this site.
When you call the bookmark, a window pops up in which you can enter a search string. The search is limited to the current site.
NB : this bookmark works with any site.
But you can also use the Google search applied just to this site, and you would be surprised by the amount of results found.
In case you don't know this trick :
In the URL bar of your browser, copy/paste this :
javascript:Qr=prompt('Search%20Site%20for','');if(Qr)location.href='http://www.google.com/search?&q=site:'+ ... escape(Qr)
Then save it as a bookmark, named for instance Google this site.
When you call the bookmark, a window pops up in which you can enter a search string. The search is limited to the current site.
NB : this bookmark works with any site.
Michel
MacOsX 12.7.5, Finale 27.4.1 & 26.3.1, Mac Mini Intel Dual Core i7 3Ghz, 16 Go Ram. Azerty kb. MOTU Midi Express XT USB, Roland Sound Canvas SC-88vl, MOTU Audio Express. 2 monitors (27"' pivot, 24'"), JW Lua, RGP Lua
MacOsX 12.7.5, Finale 27.4.1 & 26.3.1, Mac Mini Intel Dual Core i7 3Ghz, 16 Go Ram. Azerty kb. MOTU Midi Express XT USB, Roland Sound Canvas SC-88vl, MOTU Audio Express. 2 monitors (27"' pivot, 24'"), JW Lua, RGP Lua
Thanks very much michelp for the helpful post. It's useful but I'm truly interested in downloading the entire forum archives and putting them on a disc for reference in case ever needed. If I have to save the pages one at a time, then I'll do it over a period of time.
G.P.
- michelp
- Posts: 2073
- Joined: Fri Jul 25, 2003 3:35 pm
- Finale Version: 27.4.1,26.3.1, Mont.
- Operating System: Mac
I perfectly understood that you want something else.
Just an idea : there are some softwares which download whole web sites for offline viewing (Sitesucker on the Mac, etc...). Maybe one of them could help...
Not sure they work for forums, database sites...
Just an idea : there are some softwares which download whole web sites for offline viewing (Sitesucker on the Mac, etc...). Maybe one of them could help...
Not sure they work for forums, database sites...
Michel
MacOsX 12.7.5, Finale 27.4.1 & 26.3.1, Mac Mini Intel Dual Core i7 3Ghz, 16 Go Ram. Azerty kb. MOTU Midi Express XT USB, Roland Sound Canvas SC-88vl, MOTU Audio Express. 2 monitors (27"' pivot, 24'"), JW Lua, RGP Lua
MacOsX 12.7.5, Finale 27.4.1 & 26.3.1, Mac Mini Intel Dual Core i7 3Ghz, 16 Go Ram. Azerty kb. MOTU Midi Express XT USB, Roland Sound Canvas SC-88vl, MOTU Audio Express. 2 monitors (27"' pivot, 24'"), JW Lua, RGP Lua
Thanks. I've got the software on both mac and pc but I contacted the forum admin per miker's advice. Best to ask these things. Some web server will block ip addresses when there is too much and too fast download activity. So, better to find out exactly where the forum posts are and if the bulk downloading of them is approved.
G.P.
- Erik Van Geit
- Posts: 32
- Joined: Fri May 14, 2010 9:27 pm
Hi,
Are there any results about this question?
kind regards,
Erik
Are there any results about this question?
kind regards,
Erik
forum admin emailed me with a "no" to the possibility of a bulk download
I had tried it earlier with a web site downloader and the forum is setup with the topic pages as .php files
they wouldn't open normally in my browser and the downloader couldn't save them in a form that would allow the forum to be read from disk as if I were logged in and reading the pages
it would be a great service to be able to download the forums as zip files or some other archive form but a lot of work to put them in an easily readable form.
many forums are setup to discourage bulk download and the strain that would put on their servers.
I had tried it earlier with a web site downloader and the forum is setup with the topic pages as .php files
they wouldn't open normally in my browser and the downloader couldn't save them in a form that would allow the forum to be read from disk as if I were logged in and reading the pages
it would be a great service to be able to download the forums as zip files or some other archive form but a lot of work to put them in an easily readable form.
many forums are setup to discourage bulk download and the strain that would put on their servers.
G.P.
- michelp
- Posts: 2073
- Joined: Fri Jul 25, 2003 3:35 pm
- Finale Version: 27.4.1,26.3.1, Mont.
- Operating System: Mac
One thing is to downlooad the whole database, another thing is to be able to search through all the information with efficiency.
Hence my post about a better search method (see the 3rd post in this thread), which could be a reasonable way to obtain information without having to store it.
Hence my post about a better search method (see the 3rd post in this thread), which could be a reasonable way to obtain information without having to store it.
Michel
MacOsX 12.7.5, Finale 27.4.1 & 26.3.1, Mac Mini Intel Dual Core i7 3Ghz, 16 Go Ram. Azerty kb. MOTU Midi Express XT USB, Roland Sound Canvas SC-88vl, MOTU Audio Express. 2 monitors (27"' pivot, 24'"), JW Lua, RGP Lua
MacOsX 12.7.5, Finale 27.4.1 & 26.3.1, Mac Mini Intel Dual Core i7 3Ghz, 16 Go Ram. Azerty kb. MOTU Midi Express XT USB, Roland Sound Canvas SC-88vl, MOTU Audio Express. 2 monitors (27"' pivot, 24'"), JW Lua, RGP Lua