摘要: 这个是我最困扰的,也是我想了好久才想出来的。游戏中,你肯定要判断当前的路是不是可以走,当然在没有障碍物的情况下,你是可以随便走的,但是我的这个游戏里面有地图,有地图肯定就得有障碍物,我的想法是这样先把一张图片(320*240),按照1*1的大小拆分下来,用一个矩阵表示当前坐标的状态,例如(20,30,1)这三个参数分别表示X坐标,Y坐标,最后一个参数0表示可以到达,1表示不可到达。这样经过矩阵的初始化以后,就可以在逻辑上知道,当前玩家是否可以行走了。下来就是寻路了,寻路是游戏开发中非常重要的一个元素,如何找到一条最短的路径是程序需要设计的算法,由于自己在算法上也有点研究,自己也想了好久,终于想 阅读全文
posted @ 2013-05-13 00:15 史红星-shihongxing 阅读(432) 评论(0) 推荐(0) 编辑
凌晨接到hive作业异常,hive版本为1.2.1,hadoop版本apache 2.7.1,元数据存储在mysql中,异常信息如下:

Logging initialized using configuration in jar:file:/opt/apache-hive-1.2.1-bin/lib/hive-common-1.2.1.jar!/hive-log4j.properties
Exception in thread "main" java.lang.RuntimeException: java.lang.RuntimeException: Unable to instantiate org.apache.hadoop.hive.ql.metadata.SessionHiveMetaStoreClient
at org.apache.hadoop.hive.ql.session.SessionState.start(SessionState.
at org.apache.hadoop.hive.cli.CliDriver.run(CliDriver.
at org.apache.hadoop.hive.cli.CliDriver.main(CliDriver.
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.
at
at org.apache.hadoop.util.RunJar.run(RunJar.
at org.apache.hadoop.util.RunJar.main(RunJar.
Caused by: java.lang.RuntimeException: Unable to instantiate org.apache.hadoop.hive.ql.metadata.SessionHiveMetaStoreClient
at org.apache.hadoop.hive.metastore.MetaStoreUtils.newInstance(MetaStoreUtils.
at org.apache.hadoop.hive.metastore.RetryingMetaStoreClient.(RetryingMetaStoreClient.
at org.apache.hadoop.hive.metastore.RetryingMetaStoreClient.getProxy(RetryingMetaStoreClient.
at org.apache.hadoop.hive.metastore.RetryingMetaStoreClient.getProxy(RetryingMetaStoreClient.
at org.apache.hadoop.hive.ql.metadata.Hive.createMetaStoreClient(Hive.
at org.apache.hadoop.hive.ql.metadata.Hive.getMSC(Hive.
at org.apache.hadoop.hive.ql.session.SessionState.start(SessionState.
... 8 more
Caused by: java.lang.reflect.InvocationTargetException
at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.
at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.
at
at org.apache.hadoop.hive.metastore.MetaStoreUtils.newInstance(MetaStoreUtils.
... 14 more
Caused by: MetaException(message:Metastore contains multiple versions (2) [ version = 1.2.0, comment = Set by MetaStore *@*.*.*.* ] [ version = 1.2.0, comment = Set by MetaStore *@*.*.*.* ])
at org.apache.hadoop.hive.metastore.ObjectStore.getMSchemaVersion(ObjectStore.
at org.apache.hadoop.hive.metastore.ObjectStore.getMetaStoreSchemaVersion(ObjectStore.
at org.apache.hadoop.hive.metastore.ObjectStore.checkSchema(ObjectStore.
at org.apache.hadoop.hive.metastore.ObjectStore.verifySchema(ObjectStore.
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.
at
at org.apache.hadoop.hive.metastore.RawStoreProxy.invoke(RawStoreProxy.
at $Proxy9.verifySchema(Unknown Source)
at org.apache.hadoop.hive.metastore.HiveMetaStore$HMSHandler.getMS(HiveMetaStore.
at org.apache.hadoop.hive.metastore.HiveMetaStore$HMSHandler.createDefaultDB(HiveMetaStore.
at org.apache.hadoop.hive.metastore.HiveMetaStore$HMSHandler.init(HiveMetaStore.
at org.apache.hadoop.hive.metastore.RetryingHMSHandler.(RetryingHMSHandler.
at org.apache.hadoop.hive.metastore.RetryingHMSHandler.getProxy(RetryingHMSHandler.
at org.apache.hadoop.hive.metastore.HiveMetaStore.newRetryingHMSHandler(HiveMetaStore.
at org.apache.hadoop.hive.metastore.HiveMetaStoreClient.(HiveMetaStoreClient.
at org.apache.hadoop.hive.ql.metadata.SessionHiveMetaStoreClient.(SessionHiveMetaStoreClient

异常是因为在启动hive命令时会检查hive源数据中有一张VERSION表,如果元数据版本信息获取不到(原因可能是元数据库异常||网络异常||短期内作业量较多操作都会造成查询不到版本信息),这种情况下会判断hive.metastore.schema.verification属性是true还是false,为true时直接抛出MetaException,为false时打出warn警告然后插入一条version数据(这种情况下会造成多条version记录后面的作业会受影响),下面为hive-metastore包中ObjectStore类代码,标红处为造成多条version记录的代码;

 private synchronized void checkSchema() throws MetaException {
    // recheck if it got verified by another thread while we were waiting
    if (isSchemaVerified.get()) {
      return;
    }

    boolean strictValidation =
      HiveConf.getBoolVar(getConf(), HiveConf.ConfVars.METASTORE_SCHEMA_VERIFICATION);
    // read the schema version stored in metastore db
    String schemaVer = getMetaStoreSchemaVersion();
    if (schemaVer == null) {
      if (strictValidation) {
        throw new MetaException("Version information not found in metastore. ");
      } else {
        LOG.warn("Version information not found in metastore. "
            + HiveConf.ConfVars.METASTORE_SCHEMA_VERIFICATION.toString() +
            " is not enabled so recording the schema version " +
            MetaStoreSchemaInfo.getHiveSchemaVersion());
        setMetaStoreSchemaVersion(MetaStoreSchemaInfo.getHiveSchemaVersion(),
          "Set by MetaStore " + USER + "@" + HOSTNAME);
      }
    } else {
      // metastore schema version is different than Hive distribution needs
      if (schemaVer.equalsIgnoreCase(MetaStoreSchemaInfo.getHiveSchemaVersion())) {
        LOG.debug("Found expected HMS version of " + schemaVer);
      } else {
        if (strictValidation) {
          throw new MetaException("Hive Schema version "
              + MetaStoreSchemaInfo.getHiveSchemaVersion() +
              " does not match metastore's schema version " + schemaVer +
              " Metastore is not upgraded or corrupt");
        } else {
          LOG.error("Version information found in metastore differs " + schemaVer +
              " from expected schema version " + MetaStoreSchemaInfo.getHiveSchemaVersion() +
              ". Schema verififcation is disabled " +
              HiveConf.ConfVars.METASTORE_SCHEMA_VERIFICATION + " so setting version.");
          setMetaStoreSchemaVersion(MetaStoreSchemaInfo.getHiveSchemaVersion(),
            "Set by MetaStore " + USER + "@" + HOSTNAME);
        }
      }
    }
    isSchemaVerified.set(true);
    return;
  }

解决方案:

 

hive安装好后将hive-site.xml中hive.metastore.schema.verification设置为true,version获取不到时报出异常,不去插入version信息,这样本作业执行失败不会影响下游作业;

开启metastore服务,hive统一连接metastore,由守护进程启动metastore,避免大量hive脚本初始化元数据信息时获取不到版本信息;

优化hive元数据库;

 

修改观察几天通过grep "Version information not found in metastore"  hive.log发现没有再报找不到version的异常了;

 

posted @ 2019-04-24 10:16 史红星-shihongxing 阅读(578) 评论(0) 推荐(0) 编辑
摘要: 最近感觉我的服务器特别卡,打开数据库都半天,刚开始以为网咯不好也没太在意。 利用top命令: 这时候问题出来了,最高cpu占用100%,那我用啥??? 根据进程id 一看究竟,ps -ef|grep 进程id 1.CPU占用最多的前10个进程:ps auxw|head -1;ps auxw|sort 阅读全文
posted @ 2019-01-08 16:27 史红星-shihongxing 阅读(724) 评论(0) 推荐(1) 编辑
摘要: 需要LuaJIT-2.0.4.tar.gz,ngx_devel_kit,lua-nginx-module 1.下载安装LuaJIT-2.0.5.tar.gz http://luajit.org/download.html wget -c http://luajit.org/download/LuaJ 阅读全文
posted @ 2018-11-14 17:12 史红星-shihongxing 阅读(806) 评论(0) 推荐(0) 编辑
摘要: The following command fails: $ ab -n 1 localhost:8000/ ... Benchmarking localhost (be patient)...apr_socket_recv: Connection refused (111) But this one succeeds: $ ab -n 1 127.0.0.1:8000/ In /etc... 阅读全文
posted @ 2018-11-13 11:51 史红星-shihongxing 阅读(258) 评论(0) 推荐(0) 编辑
摘要: 在zkServer.sh中,增加以下参数: start) echo -n "Starting zookeeper ... " if [ -f $ZOOPIDFILE ]; then if kill -0 `cat $ZOOPIDFILE` > /dev/null 2>&1; then echo $c 阅读全文
posted @ 2018-11-12 20:27 史红星-shihongxing 阅读(1355) 评论(0) 推荐(0) 编辑
摘要: JDK: 版本:1.8 服务器默认安装1.7,所以不使用默认安装 配置环境变量 进入java的文件夹下发现bin目录,JAVA_HOME指向一个含有java可执行程序的目录(一般是在 bin/java中,此目录为/bin/java的上级目录) 让系统上的所有用户使用java(openjdk),则配置 阅读全文
posted @ 2018-11-12 10:28 史红星-shihongxing 阅读(177) 评论(0) 推荐(0) 编辑
摘要: Install zookeeper 修改配置文件 1、安装 yum install csh http://cr.yp.to/daemontools/daemontools-0.76.tar.gz下载 最后一句命令是在 ‘/etc/rc.d/rc.local’ 文件上加上了一句: csh -cf '/ 阅读全文
posted @ 2018-11-09 20:49 史红星-shihongxing 阅读(170) 评论(0) 推荐(0) 编辑
摘要: 2.清理下下缓存 [root@bastion-IDC yum.repos.d]# yum clean all [root@bastion-IDC yum.repos.d]# yum makecache //将服务器上的软件包信息缓存到本地,以提高搜索安装软件的速度[root@bastion-IDC 阅读全文
posted @ 2018-11-09 15:31 史红星-shihongxing 阅读(192) 评论(0) 推荐(0) 编辑
摘要: 服务器上的yum突然不好使用,使用yum的时候报错如下:[root@bastion-IDC src]# yum list......Could not retrieve mirrorlist http://mirrorlist.repoforge.org/el6/mirrors-rpmforge e 阅读全文
posted @ 2018-11-09 14:47 史红星-shihongxing 阅读(1360) 评论(0) 推荐(0) 编辑
摘要: 功能: 在使用memcached时候,怕因为一些不可预知的因素导致memcached进程死掉,而又不能及时的发现重启,可以通过daemontools来管理memcached的启动,当memcached死掉后系统会自动在5秒内重memcached; 1、安装 yum install csh http: 阅读全文
posted @ 2018-11-09 11:21 史红星-shihongxing 阅读(696) 评论(0) 推荐(0) 编辑
点击右上角即可分享
微信分享提示