google地图大规模下载防屏蔽手段

最近需要大规模下载google地图(需要几十万张图片的规模),该怎么办呢?

首先要有下载程序嘛,工欲善其事,必先利其器

c#代码可以很容易的实现

void downloadFile(string url,string localPath)
        {
            try
            {
                var request = HttpWebRequest.Create(url);
                request.Headers.Add("cookie", "khcookie=fzwq2qFFXpeBQ3lUygacJVzS8pSXFknR02LwWA; rememberme=true; PREF=ID=092f8e3b732e1bfd:U=10b13387c44535ee:FF=0:LD=en:NW=1:CR=2:TM=1313217539:LM=1319457716:GM=1:IG=4:S=n36T5WxY0Ky8lO3j; HSID=AvfT-czml1Jj1MlTe; APISID=XOphmR9lEnjRRpJN/Alsn0LcocxoR3x9mV; NID=52=KITQ783WQywC29reXSuqSRF4ij5-Lv8T5_4CjB6QRpmGglTa0rA6xPFifJ86DRshBulJdLm8mjlRCYCZ7jG5qqjDiHjtCl17-7SGRNQV-ZZd3LsgcFXzIRPUYqyEtZcc; SID=DQAAALkAAAAB6wVsuiqESra9YBnbPy6yG2P0aamI8yo_37UD0HYC71gWNiqq6x5cDbjQoBSa2dIIZxiZJjElLlvq08yjtdGI1uiIDenhF9CKnOkhC7GJoNR_jCt4gYLufoWX5djLI-VUmjpSZ0byfp91bZ5mm9AY-qMlUmRmnu32Ld5mYkJCd-sAsDpdIXH0gaIAomKbW0p1YTl6vzw9bd_Cx0AoBuVBt_8nIxXuGSn2Tb7usk8afi0T6ADeGZl52ca7w-_1f0Y; GDSESS=ID=09a0ec5363532e8c:TM=1320337601:C=c:IP=****-:S=ADSvE-eKmoE7YIepKdB1lL6v9wdv-qNtUQ");
                var response = request.GetResponse();
                long totalBytes = response.ContentLength;
                //proDownload.Maximum = (int)totalBytes;
                var si = response.GetResponseStream();
                var so = new FileStream(localPath, FileMode.Create);
                long downloadedBytes = 0;
                byte[] buffer = new byte[1024*10];
 
                int readSize;
                do
                {
                    readSize = si.Read(buffer, 0, (int)buffer.Length);
                    downloadedBytes += readSize;
                    so.Write(buffer, 0, readSize);
                    //proDownload.Value = (int)downloadedBytes;
                    //Application.DoEvents();
                } while (readSize>0);
                si.Close();
                so.Close();
            }
            catch (System.Exception ex)
            {
                if (ex.Message.Contains("503"))
                    throw ex;
                if (ex.Message.Contains("404"))
                    return;
                txtStatus.Text += string.Format("{0}",ex.Message);
            }
        }

注意,代码里有一个****的东西,代表了我自己的IP,故给隐掉了,至于那一段有什么用,暂且不表

其他的部分都很好理解,创建一个HttpWebRequest对象,访问图片网址,用10k的buffer来下载,保存到本地

其中有两个比较重要的异常,一个是503,那很不幸,你被屏蔽了;另一个是404,你懂得

那么怎么获得google地图的图片地址呢?google提供给了我们一个静态图片api(http://code.google.com/apis/maps/documentation/staticmaps/),可惜每天的访问限制是1000次,如果要下载20万幅的话,恐怕得等上一年半载的,是可忍孰不可忍?我们必须另觅蹊径.

ok,发动我们强大的chrome技能吧!

打开google map,启动chrome 开发者工具,切换到Resources选项卡,在左边的树形文件夹中找到形如 v=**&x=**&y=**&z=** 的东西,不要急,慢慢翻,随便点开一个看看,呵呵,这不就是我们寻找的玩意嘛~