[open-bibliography] OCLC's license for FAST

Thomas Krichel krichel at openlib.org
Thu Jan 5 10:56:24 UTC 2012


  William Waites writes
> On Wed, 4 Jan 2012 23:49:44 +0100, Thomas Krichel <krichel at openlib.org> said:
> 
>     >> Easy peasy. Script below. Warning: it's not a very polite
>     >> crawler, or
> 
>     pkr1>   I added a sleep 10 between requests (I tihnk) just to make
>     pkr1> it a bit more polite.
> 
> As I understand it there are about 1.6 million records so with a 10
> second delay we can expect it to finish in about six months...

  But your script stores all files in one directory, so I thought
  there will be at most 100k file to download. 

  Can you send me your public key, I will authorize it for the 
  account and you can fix?


  Cheers,

  Thomas Krichel                    http://openlib.org/home/krichel
                                      http://authorprofile.org/pkr1
                                               skype: thomaskrichel




More information about the open-bibliography mailing list