做了个多语种网站 不能被GOOGLE,yahoo收录,baidu可以收录 终于找到问题

做了个多语种网站
在BaseForm类(窗体父类)中 重写InitializeCulture以设置浏览器默认语言
 protected override void InitializeCulture()
        {
           CurLanguage  = Request.UserLanguages[0].ToLower();
           string UserCulture = CurLanguage;
            if (UserCulture != "")
            {
                //根据Session的值重新绑定语言代码
                Thread.CurrentThread.CurrentUICulture = new CultureInfo(UserCulture);
                Thread.CurrentThread.CurrentCulture = CultureInfo.CreateSpecificCulture(UserCulture);
            }
        }

///当前页面语种(封装对Session["PreferredCulture"] 的读写)
 public string CurLanguage
        {
            set
            {
                if (Session["PreferredCulture"] == null)
                {
                    Session.Add("PreferredCulture", value);
                }
                else
                {
                    Session["PreferredCulture"] = value;
                }
            }
            get
            {
                if (Session["PreferredCulture"] == null)
                {
                    Session.Add("PreferredCulture", "");
                    return Utilities.ObjectHelper.GetString(Session["PreferredCulture"]);
                }
                else
                {
                    return Utilities.ObjectHelper.GetString(Session["PreferredCulture"]);
                }
            }
        }

网站发布后没有问题,但是过了半个月客户反映GOOGLE没办法收录,BAIDU可以收录
百思不得其解
一日有了灵感,查看了一下该网站的IIS访问日志,发现了一条记录
 /index.aspx - 80 - 209.85.238.25 AdsBot-Google+(+http://www.google.com/adsbot.html) 500 0 0 竟然有错误

然后使用GOOGLE的网站管理员工具进行测试,https://www.google.com/webmasters,都不能通过,还是提示500错误

于是自己使用fiddler工具对页面进行请求,发现了问题所在

通过fiddler发出的请示,其Request.UserLanguages为空!!!

找到了问题点就好解决了,在取值时加入空值判断,

 if (Request.UserLanguages == null )
                {
                    CurLanguage = Helpers.GetConfigToString("DefaultLanguage"); ;

                }
                else {
                    if(!string.IsNullOrEmpty(Request.UserLanguages[0])){
                        CurLanguage  = Request.UserLanguages[0].ToLower();
                    }
               
                }

然后重新用fiddler,GOOGLE测试,问题解决,返回期待的"200"