east – 第75页 – gitweixin

Java 4月 24,2020

lnmp解决mysql The server quit without updating PID file

lnmp安装包，挂个小网站，mysql跑了几年，一直没去动它。后来有时mysql 挂掉了，偷懒写个自动重启脚本，又继续安稳几个月时间。今天突然发现mysql挂了，重启了报“The server quit without updating PID file”。按网上方法试了还是没解决，后来看到有点说磁盘满了，“df -l”一查看，果真满了，请了日志后果然能重启mysql正常了。

后来又遇到这个问题，磁盘没满。看到错误日志有
“Plugin ‘FEDERATED’ is disabled”

在my.cnf文件中的 [mysqld] 中增加一行
tmpdir=”/usr/local/mysql/var”

同时增加权限 chown mysql:mysql /usr/local/mysql/var

运行service mysql start还是出错，查看错误日志看到

InnoDB: Database page corruption on disk or a failed

添加下面内容:

innodb_force_recovery=1

再启动mysql

则可以正常启动。

作者 east

bug清单, Hadoop 3月 2,2020

hadoop DataNode服务无法启动解决（报java.net.BindException:地址已在使用）

启动DataNode服务后马上又Shutdown，在操作系统没看到有DataNode的日志（可能是服务启动失败，自动删除了日志文件），幸好在界面上可以查看报错的日志：

点开报错信息，可以看到如下信息：

HDFS的端口为50010，但是使用netstat -ntulp | grep 50010查看不到此端口。

分析：

原因：当应用程序崩溃后，它会留下一个滞留的socket，以便能够提前重用socket，当尝试绑定socket并重用它，你需要将socket的flag设置为SO_REUSEADDR，但是HDFS不是这么做的。解决办法是使用设置SO_REUSEADDR的应用程序绑定到这个端口，然后停止这个应用程序。可以使用netcat工具实现。解决办法：安装nc工具，使用nc工具占用50010端口，然后关闭nc服务，再次启动DataNode后正常。

参考链接：http://www.nosql.se/2013/10/hadoop-hdfs-datanode-java-net-bindexception-address-already-in-use/参考文字：

    After an application crashes it might leave a lingering socket, so to reuse that socket early you need to set the socket flag SO_REUSEADDR when attempting to bind to it to be allowed to reuse it. The HDFS datanode doesn’t do that, and I didn’t want to restart the HBase regionserver (which was locking the socket with a connection it hadn’t realized was dead).The solution was to bind to the port with an application that sets SO_REUSEADDR andthen stop that application, I used netcat for that:# nc -l 50010

2017-02-17 20:54:52,250 INFO org.apache.hadoop.hdfs.server.datanode.DataNode: Shutdown complete.2017-02-17 20:54:52,251 FATAL org.apache.hadoop.hdfs.server.datanode.DataNode: Exception in secureMainjava.net.BindException: Address already in use	at sun.nio.ch.Net.bind0(Native Method)	at sun.nio.ch.Net.bind(Net.java:444)	at sun.nio.ch.Net.bind(Net.java:436)	at sun.nio.ch.ServerSocketChannelImpl.bind(ServerSocketChannelImpl.java:214)	at sun.nio.ch.ServerSocketAdaptor.bind(ServerSocketAdaptor.java:74)	at com.cloudera.io.netty.channel.socket.nio.NioServerSocketChannel.doBind(NioServerSocketChannel.java:125)	at com.cloudera.io.netty.channel.AbstractChannel$AbstractUnsafe.bind(AbstractChannel.java:475)	at com.cloudera.io.netty.channel.DefaultChannelPipeline$HeadContext.bind(DefaultChannelPipeline.java:1021)	at com.cloudera.io.netty.channel.AbstractChannelHandlerContext.invokeBind(AbstractChannelHandlerContext.java:455)	at com.cloudera.io.netty.channel.AbstractChannelHandlerContext.bind(AbstractChannelHandlerContext.java:440)	at com.cloudera.io.netty.channel.DefaultChannelPipeline.bind(DefaultChannelPipeline.java:844)	at com.cloudera.io.netty.channel.AbstractChannel.bind(AbstractChannel.java:194)	at com.cloudera.io.netty.bootstrap.AbstractBootstrap$2.run(AbstractBootstrap.java:340)	at com.cloudera.io.netty.util.concurrent.SingleThreadEventExecutor.runAllTasks(SingleThreadEventExecutor.java:380)	at com.cloudera.io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:357)	at com.cloudera.io.netty.util.concurrent.SingleThreadEventExecutor$2.run(SingleThreadEventExecutor.java:116)	at com.cloudera.io.netty.util.concurrent.DefaultThreadFactory$DefaultRunnableDecorator.run(DefaultThreadFactory.java:137)	at java.lang.Thread.run(Thread.java:745)2017-02-17 20:54:52,262 INFO org.apache.hadoop.util.ExitUtil: Exiting with status 12017-02-17 20:54:52,264 INFO org.apache.hadoop.hdfs.server.datanode.DataNode: SHUTDOWN_MSG: /************************************************************SHUTDOWN_MSG: Shutting down DataNode at cdh1/192.168.5.78

作者 east

Java 3月 2,2020

springboot导出excel工具类

try{
    //捕获内存缓冲区的数据，转换为字节数组
    ByteArrayOutputStream out  = new ByteArrayOutputStream();
    workbook.write(out);
    //获取内存缓冲中的数据
    byte[] content = out.toByteArray();
    //将字节数据转化为输入流
    InputStream in = new ByteArrayInputStream(content);
    //通过调用reset()方法可以重新定位
    response.reset();
    //JSONP 解决跨域问题
    response.addHeader("Access-Control-Allow-Origin", "*");
    response.addHeader("Access-Control-Allow-Methods","GET,POST,PUT,DELETE,OPTIONS");
    response.addHeader("Access-Control-Allow-Headers", "WWW-Authenticate,Authorization,Set-Cookie," +
            "X-Requested-With,Accept-Version,Content-Length,Content-Type,Date,X-Api-Version,name");
    response.addHeader("Access-Control-Allow-Credentials", "true");
 //   response.setContentType("application/octet-stream");
    //如果文件名是英文名不需要加编码格式，如果是中文名需要添加"ios-8859-1"防止乱码
    response.setHeader("Content-Disposition", "attachment;filename=" +
            new String((fileName + ".xls").getBytes("gb2312"), "iso-8859-1"));
    response.setHeader("Content-Length", "" + content.length);
    response.setContentType("application/vnd.ms-excel;charset=UTF-8");
    ServletOutputStream outputStream = response.getOutputStream();
    BufferedInputStream bis = new BufferedInputStream(in);
    BufferedOutputStream bos = new BufferedOutputStream(outputStream);
    byte[] buff = new byte[8192];
    int bytesRead;
    while (-1 != (bytesRead = bis.read(buff, 0, buff.length))){
        bos.write(buff, 0, bytesRead);
    }
    bis.close();
    bos.close();
    outputStream.flush();
    outputStream.close();

}catch (IOException ex){
    ex.printStackTrace();
}

作者 east

bug清单 2月 23,2020

配置nginx ssl证书问题排查

小程序的ssl证书过期了，重新生成证书，替换证书后，在小程序还提示连接超时，看网上说https不安全也有这个提示。

访问访问，果然提示证书过期了。

检查nginx配置文件

server {    
 listen 80;
 server_name bjubi.com;// 你的域名    
 rewrite ^(.*)$ https://$host$1 permanent;// 把http的域名请求转成https
 }

server {
  listen 443;   
  server_name bjubi.com; // 你的域名    
  ssl on;   
  root /var/www/bjubi.com; // 前台文件存放文件夹，可改成别的   
  index index.html index.htm;// 上面配置的文件夹里面的index.html         ssl_certificate  cert/214292799730473.pem;// 改成你的证书的名字     ssl_certificate_key cert/214292799730473.key;// 你的证书的名字     ssl_session_timeout 5m;  
   ssl_ciphers ECDHE-RSA-AES128-GCM-SHA256:ECDHE:ECDH:AES:HIGH:!NULL:!aNULL:!MD5:!ADH:!RC4;   
  ssl_protocols TLSv1 TLSv1.1 TLSv1.2;   
  ssl_prefer_server_ciphers on;   
  location / {  
       index index.html index.htm;
     } 
} 
觉得配置没问题，继续排查

 
配置完成后，检查一下nginx配置文件是否可用，有successful表示可用。
$ nginx -t // 检查nginx配置文件 
返回结果如下：
nginx: the configuration file /usr/local/nginx/conf/nginx.conf syntax is ok
 nginx: configuration file /usr/local/nginx/conf/nginx.conf test is successful
觉得配置应该没问题。

 为了使配置生效
nginx -s reload // 使配置生效 
shell提示：  nginx: [alert] kill(1617, 1) failed (3: No such process)
果然出问题了。


 
[root@localhost /]# whereis ngnix
ngnix:[root@localhost/]# 
[root@localhost /]# find / -name nginx 
/usr/local/src/nginx/sbin/nginx
[root@localhost /]# find / -name nginx.conf
/usr/local/nginx
 /usr/local/nginx/sbin/nginx
 /usr/bin/nginx
 /etc/rc.d/init.d/nginx
[root@localhost /]# /usr/local/src/nginx/sbin/nginx -c nginx: [emerg] bind() to 0.0.0.0:80 failed (98: Address already in use)
 nginx: [emerg] bind() to 0.0.0.0:443 failed (98: Address already in use)
 nginx: [emerg] bind() to 0.0.0.0:80 failed (98: Address already in use)
 nginx: [emerg] bind() to 0.0.0.0:443 failed (98: Address already in use)
 nginx: [emerg] bind() to 0.0.0.0:80 failed (98: Address already in use)
 nginx: [emerg] bind() to 0.0.0.0:443 failed (98: Address already in use)
 nginx: [emerg] bind() to 0.0.0.0:80 failed (98: Address already in use)
 nginx: [emerg] bind() to 0.0.0.0:443 failed (98: Address already in use)
 nginx: [emerg] bind() to 0.0.0.0:80 failed (98: Address already in use)
 nginx: [emerg] bind() to 0.0.0.0:443 failed (98: Address already in use)
 nginx: [emerg] still could not bind()

端口被绑定了，需要先kill掉占用的线程
 
netstat -ntlp 

shell返回结果如下：
Active Internet connections (only servers)
 Proto Recv-Q Send-Q Local Address           Foreign Address         State       PID/Program name    
 tcp        0      0 0.0.0.0:3306            0.0.0.0:*               LISTEN      28256/mysqld        
 tcp        0      0 0.0.0.0:80              0.0.0.0:*               LISTEN      3301/nginx: worker  
 tcp        0      0 0.0.0.0:21              0.0.0.0:*               LISTEN      919/pure-ftpd       
 tcp        0      0 0.0.0.0:22              0.0.0.0:*               LISTEN      2565/sshd           
 tcp        0      0 0.0.0.0:443             0.0.0.0:*               LISTEN      3301/nginx: worker  
 tcp6       0      0 :::21                   :::*                    LISTEN      919/pure-ftpd   

使用命令 kill -9 3301
再制定配置 /usr/local/nginx/sbin/nginx -c /usr/local/nginx/conf/nginx.conf
 nginx -s reload            
重启nginx，果然证书生效了。

作者 east

spring 2月 6,2020

凯利公式源代码

凯利公式如下：

仓位 =（odds *pwin-q）/b

　　odds = 赔率（赔率=期望盈利÷可能亏损=2美元盈利÷1美元亏损，赔率就是2了）

　　pwin = 成功概率（抛硬币正反面都是50%的概率）

　　q = 失败概率（也就是 1-p，赌局中也是50%了）


public class kellyUtil {

    /**
     *   凯利公式
     * @param pwin  胜率
     * @param odds  赔率
     * @return
     */
    public static double kelly(double pwin,double odds){
        return (odds * pwin + pwin - 1)/odds;
    }

    public static double kellyV2(double pwin,double odds, double lossRate){
        return (odds * pwin + pwin - 1)/(odds * lossRate);
    }

    /**
     * 巴菲特版仓位管理
     * @param pwin
     * @return
     */
    public static double buffett(double pwin){
        return 2 * pwin - 1;
    }

    public static void main(String[] args) {
        double odds = kelly(0.5, 3.0);
      //  double odds = kellyV2(0.5, 3.0,1.5);
        System.out.println(("仓位:" + odds * 100 + "%"));
    }
}

作者 east

spring 1月 14,2020

maven 增加本地libs依赖的完美方案

1、首先新建libs文件夹，把外部依赖的jar放进去。

2、在pom文件把外部依赖jar文件导进去

<dependency>
    <groupId>org.codehaus.stax2</groupId>
    <artifactId>stax2</artifactId>
    <version>3.1.4</version>
    <scope>system</scope>
    <systemPath>${project.basedir}/libs/stax2-api-3.1.4.jar</systemPath>
</dependency>

其中groupId、artifactId和version都可以自己定义 scope是system，和provided类似，只是jar包本地提供，这种方式有个缺点，那就是在项目打成jar或war包的时候因为scope是system，只在编译的时候能用，install的时候不会打进去。

在pom中给spring boot的打包插件设置一下includeSystemScope参数即可?

<build>
<plugins>
<plugin>
<groupId>org.springframework.boot</groupId>
<artifactId>spring-boot-maven-plugin</artifactId>
<configuration>
<includeSystemScope>true</includeSystemScope>
</configuration>
</plugin>
</plugins>
</build>

作者 east

namenode 日志提示 storage directory does not exist or is not accessible