[open-bibliography] OCLC's license for FAST
Thomas Krichel
krichel at openlib.org
Thu Jan 5 10:56:24 UTC 2012
William Waites writes
> On Wed, 4 Jan 2012 23:49:44 +0100, Thomas Krichel <krichel at openlib.org> said:
>
> >> Easy peasy. Script below. Warning: it's not a very polite
> >> crawler, or
>
> pkr1> I added a sleep 10 between requests (I tihnk) just to make
> pkr1> it a bit more polite.
>
> As I understand it there are about 1.6 million records so with a 10
> second delay we can expect it to finish in about six months...
But your script stores all files in one directory, so I thought
there will be at most 100k file to download.
Can you send me your public key, I will authorize it for the
account and you can fix?
Cheers,
Thomas Krichel http://openlib.org/home/krichel
http://authorprofile.org/pkr1
skype: thomaskrichel
More information about the open-bibliography
mailing list