网络爬虫缺少安全证书的报错

2024-06-07 08:58

本文主要是介绍网络爬虫缺少安全证书的报错,希望对大家解决编程问题提供一定的参考价值,需要的开发者们随着小编来一起学习吧!

解决 sun.security.validator.ValidatorException: PKIX path building failed

 

今天用java HttpClients写爬虫在访问某Https站点报如下错误:

 

sun.security.validator.ValidatorException: PKIX path building failed...

 

网上找了找解决方案,看到这篇博客:http://www.oschina.net/question/12_19249

http://www.oschina.net/code/snippet_12_4092

 

经测试,解决问题,这里做个记录。

 

复制代码
/** Copyright 2006 Sun Microsystems, Inc.  All Rights Reserved.** Redistribution and use in source and binary forms, with or without* modification, are permitted provided that the following conditions* are met:**   - Redistributions of source code must retain the above copyright*     notice, this list of conditions and the following disclaimer.**   - Redistributions in binary form must reproduce the above copyright*     notice, this list of conditions and the following disclaimer in the*     documentation and/or other materials provided with the distribution.**   - Neither the name of Sun Microsystems nor the names of its*     contributors may be used to endorse or promote products derived*     from this software without specific prior written permission.** THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS* IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO,* THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR* PURPOSE ARE DISCLAIMED.  IN NO EVENT SHALL THE COPYRIGHT OWNER OR* CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL,* EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO,* PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR* PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF* LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT (INCLUDING* NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF THIS* SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.*/import java.io.*;
import java.net.URL;import java.security.*;
import java.security.cert.*;import javax.net.ssl.*;public class InstallCert {public static void main(String[] args) throws Exception {String host;int port;char[] passphrase;if ((args.length == 1) || (args.length == 2)) {String[] c = args[0].split(":");host = c[0];port = (c.length == 1) ? 443 : Integer.parseInt(c[1]);String p = (args.length == 1) ? "changeit" : args[1];passphrase = p.toCharArray();} else {System.out.println("Usage: java InstallCert <host>[:port] [passphrase]");return;}File file = new File("jssecacerts");if (file.isFile() == false) {char SEP = File.separatorChar;File dir = new File(System.getProperty("java.home") + SEP+ "lib" + SEP + "security");file = new File(dir, "jssecacerts");if (file.isFile() == false) {file = new File(dir, "cacerts");}}System.out.println("Loading KeyStore " + file + "...");InputStream in = new FileInputStream(file);KeyStore ks = KeyStore.getInstance(KeyStore.getDefaultType());ks.load(in, passphrase);in.close();SSLContext context = SSLContext.getInstance("TLS");TrustManagerFactory tmf =TrustManagerFactory.getInstance(TrustManagerFactory.getDefaultAlgorithm());tmf.init(ks);X509TrustManager defaultTrustManager = (X509TrustManager)tmf.getTrustManagers()[0];SavingTrustManager tm = new SavingTrustManager(defaultTrustManager);context.init(null, new TrustManager[] {tm}, null);SSLSocketFactory factory = context.getSocketFactory();System.out.println("Opening connection to " + host + ":" + port + "...");SSLSocket socket = (SSLSocket)factory.createSocket(host, port);socket.setSoTimeout(10000);try {System.out.println("Starting SSL handshake...");socket.startHandshake();socket.close();System.out.println();System.out.println("No errors, certificate is already trusted");} catch (SSLException e) {System.out.println();e.printStackTrace(System.out);}X509Certificate[] chain = tm.chain;if (chain == null) {System.out.println("Could not obtain server certificate chain");return;}BufferedReader reader =new BufferedReader(new InputStreamReader(System.in));System.out.println();System.out.println("Server sent " + chain.length + " certificate(s):");System.out.println();MessageDigest sha1 = MessageDigest.getInstance("SHA1");MessageDigest md5 = MessageDigest.getInstance("MD5");for (int i = 0; i < chain.length; i++) {X509Certificate cert = chain[i];System.out.println(" " + (i + 1) + " Subject " + cert.getSubjectDN());System.out.println("   Issuer  " + cert.getIssuerDN());sha1.update(cert.getEncoded());System.out.println("   sha1    " + toHexString(sha1.digest()));md5.update(cert.getEncoded());System.out.println("   md5     " + toHexString(md5.digest()));System.out.println();}System.out.println("Enter certificate to add to trusted keystore or 'q' to quit: [1]");String line = reader.readLine().trim();int k;try {k = (line.length() == 0) ? 0 : Integer.parseInt(line) - 1;} catch (NumberFormatException e) {System.out.println("KeyStore not changed");return;}X509Certificate cert = chain[k];String alias = host + "-" + (k + 1);ks.setCertificateEntry(alias, cert);OutputStream out = new FileOutputStream("jssecacerts");ks.store(out, passphrase);out.close();System.out.println();System.out.println(cert);System.out.println();System.out.println("Added certificate to keystore 'jssecacerts' using alias '"+ alias + "'");}private static final char[] HEXDIGITS = "0123456789abcdef".toCharArray();private static String toHexString(byte[] bytes) {StringBuilder sb = new StringBuilder(bytes.length * 3);for (int b : bytes) {b &= 0xff;sb.append(HEXDIGITS[b >> 4]);sb.append(HEXDIGITS[b & 15]);sb.append(' ');}return sb.toString();}private static class SavingTrustManager implements X509TrustManager {private final X509TrustManager tm;private X509Certificate[] chain;SavingTrustManager(X509TrustManager tm) {this.tm = tm;}public X509Certificate[] getAcceptedIssuers() {throw new UnsupportedOperationException();}public void checkClientTrusted(X509Certificate[] chain, String authType)throws CertificateException {throw new UnsupportedOperationException();}public void checkServerTrusted(X509Certificate[] chain, String authType)throws CertificateException {this.chain = chain;tm.checkServerTrusted(chain, authType);}}}
复制代码

编译:javac InstallCert.java 
运行:java InstallCert ecc.fedora.redhat.com

 

C:\Program Files\Java\jdk1.8.0_74\jre\lib\security

这篇关于网络爬虫缺少安全证书的报错的文章就介绍到这儿,希望我们推荐的文章对编程师们有所帮助!



http://www.chinasem.cn/article/1038719

相关文章

解决SpringBoot启动报错:Failed to load property source from location 'classpath:/application.yml'

《解决SpringBoot启动报错:Failedtoloadpropertysourcefromlocationclasspath:/application.yml问题》这篇文章主要介绍... 目录在启动SpringBoot项目时报如下错误原因可能是1.yml中语法错误2.yml文件格式是GBK总结在启动S

idea maven编译报错Java heap space的解决方法

《ideamaven编译报错Javaheapspace的解决方法》这篇文章主要为大家详细介绍了ideamaven编译报错Javaheapspace的相关解决方法,文中的示例代码讲解详细,感兴趣的... 目录1.增加 Maven 编译的堆内存2. 增加 IntelliJ IDEA 的堆内存3. 优化 Mave

Python从零打造高安全密码管理器

《Python从零打造高安全密码管理器》在数字化时代,每人平均需要管理近百个账号密码,本文将带大家深入剖析一个基于Python的高安全性密码管理器实现方案,感兴趣的小伙伴可以参考一下... 目录一、前言:为什么我们需要专属密码管理器二、系统架构设计2.1 安全加密体系2.2 密码强度策略三、核心功能实现详解

如何解决mmcv无法安装或安装之后报错问题

《如何解决mmcv无法安装或安装之后报错问题》:本文主要介绍如何解决mmcv无法安装或安装之后报错问题,具有很好的参考价值,希望对大家有所帮助,如有错误或未考虑完全的地方,望不吝赐教... 目录mmcv无法安装或安装之后报错问题1.当我们运行YOwww.chinasem.cnLO时遇到2.找到下图所示这里3.

浅谈配置MMCV环境,解决报错,版本不匹配问题

《浅谈配置MMCV环境,解决报错,版本不匹配问题》:本文主要介绍浅谈配置MMCV环境,解决报错,版本不匹配问题,具有很好的参考价值,希望对大家有所帮助,如有错误或未考虑完全的地方,望不吝赐教... 目录配置MMCV环境,解决报错,版本不匹配错误示例正确示例总结配置MMCV环境,解决报错,版本不匹配在col

Linux系统配置NAT网络模式的详细步骤(附图文)

《Linux系统配置NAT网络模式的详细步骤(附图文)》本文详细指导如何在VMware环境下配置NAT网络模式,包括设置主机和虚拟机的IP地址、网关,以及针对Linux和Windows系统的具体步骤,... 目录一、配置NAT网络模式二、设置虚拟机交换机网关2.1 打开虚拟机2.2 管理员授权2.3 设置子

揭秘Python Socket网络编程的7种硬核用法

《揭秘PythonSocket网络编程的7种硬核用法》Socket不仅能做聊天室,还能干一大堆硬核操作,这篇文章就带大家看看Python网络编程的7种超实用玩法,感兴趣的小伙伴可以跟随小编一起... 目录1.端口扫描器:探测开放端口2.简易 HTTP 服务器:10 秒搭个网页3.局域网游戏:多人联机对战4.

SpringBoot启动报错的11个高频问题排查与解决终极指南

《SpringBoot启动报错的11个高频问题排查与解决终极指南》这篇文章主要为大家详细介绍了SpringBoot启动报错的11个高频问题的排查与解决,文中的示例代码讲解详细,感兴趣的小伙伴可以了解一... 目录1. 依赖冲突:NoSuchMethodError 的终极解法2. Bean注入失败:No qu

springboot报错Invalid bound statement (not found)的解决

《springboot报错Invalidboundstatement(notfound)的解决》本文主要介绍了springboot报错Invalidboundstatement(not... 目录一. 问题描述二.解决问题三. 添加配置项 四.其他的解决方案4.1 Mapper 接口与 XML 文件不匹配

SpringBoot使用OkHttp完成高效网络请求详解

《SpringBoot使用OkHttp完成高效网络请求详解》OkHttp是一个高效的HTTP客户端,支持同步和异步请求,且具备自动处理cookie、缓存和连接池等高级功能,下面我们来看看SpringB... 目录一、OkHttp 简介二、在 Spring Boot 中集成 OkHttp三、封装 OkHttp