网络爬虫缺少安全证书的报错

2024-06-07 08:58

本文主要是介绍网络爬虫缺少安全证书的报错,希望对大家解决编程问题提供一定的参考价值,需要的开发者们随着小编来一起学习吧!

解决 sun.security.validator.ValidatorException: PKIX path building failed

 

今天用java HttpClients写爬虫在访问某Https站点报如下错误:

 

sun.security.validator.ValidatorException: PKIX path building failed...

 

网上找了找解决方案,看到这篇博客:http://www.oschina.net/question/12_19249

http://www.oschina.net/code/snippet_12_4092

 

经测试,解决问题,这里做个记录。

 

复制代码
/** Copyright 2006 Sun Microsystems, Inc.  All Rights Reserved.** Redistribution and use in source and binary forms, with or without* modification, are permitted provided that the following conditions* are met:**   - Redistributions of source code must retain the above copyright*     notice, this list of conditions and the following disclaimer.**   - Redistributions in binary form must reproduce the above copyright*     notice, this list of conditions and the following disclaimer in the*     documentation and/or other materials provided with the distribution.**   - Neither the name of Sun Microsystems nor the names of its*     contributors may be used to endorse or promote products derived*     from this software without specific prior written permission.** THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS* IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO,* THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR* PURPOSE ARE DISCLAIMED.  IN NO EVENT SHALL THE COPYRIGHT OWNER OR* CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL,* EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO,* PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR* PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF* LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT (INCLUDING* NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF THIS* SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.*/import java.io.*;
import java.net.URL;import java.security.*;
import java.security.cert.*;import javax.net.ssl.*;public class InstallCert {public static void main(String[] args) throws Exception {String host;int port;char[] passphrase;if ((args.length == 1) || (args.length == 2)) {String[] c = args[0].split(":");host = c[0];port = (c.length == 1) ? 443 : Integer.parseInt(c[1]);String p = (args.length == 1) ? "changeit" : args[1];passphrase = p.toCharArray();} else {System.out.println("Usage: java InstallCert <host>[:port] [passphrase]");return;}File file = new File("jssecacerts");if (file.isFile() == false) {char SEP = File.separatorChar;File dir = new File(System.getProperty("java.home") + SEP+ "lib" + SEP + "security");file = new File(dir, "jssecacerts");if (file.isFile() == false) {file = new File(dir, "cacerts");}}System.out.println("Loading KeyStore " + file + "...");InputStream in = new FileInputStream(file);KeyStore ks = KeyStore.getInstance(KeyStore.getDefaultType());ks.load(in, passphrase);in.close();SSLContext context = SSLContext.getInstance("TLS");TrustManagerFactory tmf =TrustManagerFactory.getInstance(TrustManagerFactory.getDefaultAlgorithm());tmf.init(ks);X509TrustManager defaultTrustManager = (X509TrustManager)tmf.getTrustManagers()[0];SavingTrustManager tm = new SavingTrustManager(defaultTrustManager);context.init(null, new TrustManager[] {tm}, null);SSLSocketFactory factory = context.getSocketFactory();System.out.println("Opening connection to " + host + ":" + port + "...");SSLSocket socket = (SSLSocket)factory.createSocket(host, port);socket.setSoTimeout(10000);try {System.out.println("Starting SSL handshake...");socket.startHandshake();socket.close();System.out.println();System.out.println("No errors, certificate is already trusted");} catch (SSLException e) {System.out.println();e.printStackTrace(System.out);}X509Certificate[] chain = tm.chain;if (chain == null) {System.out.println("Could not obtain server certificate chain");return;}BufferedReader reader =new BufferedReader(new InputStreamReader(System.in));System.out.println();System.out.println("Server sent " + chain.length + " certificate(s):");System.out.println();MessageDigest sha1 = MessageDigest.getInstance("SHA1");MessageDigest md5 = MessageDigest.getInstance("MD5");for (int i = 0; i < chain.length; i++) {X509Certificate cert = chain[i];System.out.println(" " + (i + 1) + " Subject " + cert.getSubjectDN());System.out.println("   Issuer  " + cert.getIssuerDN());sha1.update(cert.getEncoded());System.out.println("   sha1    " + toHexString(sha1.digest()));md5.update(cert.getEncoded());System.out.println("   md5     " + toHexString(md5.digest()));System.out.println();}System.out.println("Enter certificate to add to trusted keystore or 'q' to quit: [1]");String line = reader.readLine().trim();int k;try {k = (line.length() == 0) ? 0 : Integer.parseInt(line) - 1;} catch (NumberFormatException e) {System.out.println("KeyStore not changed");return;}X509Certificate cert = chain[k];String alias = host + "-" + (k + 1);ks.setCertificateEntry(alias, cert);OutputStream out = new FileOutputStream("jssecacerts");ks.store(out, passphrase);out.close();System.out.println();System.out.println(cert);System.out.println();System.out.println("Added certificate to keystore 'jssecacerts' using alias '"+ alias + "'");}private static final char[] HEXDIGITS = "0123456789abcdef".toCharArray();private static String toHexString(byte[] bytes) {StringBuilder sb = new StringBuilder(bytes.length * 3);for (int b : bytes) {b &= 0xff;sb.append(HEXDIGITS[b >> 4]);sb.append(HEXDIGITS[b & 15]);sb.append(' ');}return sb.toString();}private static class SavingTrustManager implements X509TrustManager {private final X509TrustManager tm;private X509Certificate[] chain;SavingTrustManager(X509TrustManager tm) {this.tm = tm;}public X509Certificate[] getAcceptedIssuers() {throw new UnsupportedOperationException();}public void checkClientTrusted(X509Certificate[] chain, String authType)throws CertificateException {throw new UnsupportedOperationException();}public void checkServerTrusted(X509Certificate[] chain, String authType)throws CertificateException {this.chain = chain;tm.checkServerTrusted(chain, authType);}}}
复制代码

编译:javac InstallCert.java 
运行:java InstallCert ecc.fedora.redhat.com

 

C:\Program Files\Java\jdk1.8.0_74\jre\lib\security

这篇关于网络爬虫缺少安全证书的报错的文章就介绍到这儿,希望我们推荐的文章对编程师们有所帮助!



http://www.chinasem.cn/article/1038719

相关文章

使用Vue.js报错:ReferenceError: “Vue is not defined“ 的原因与解决方案

《使用Vue.js报错:ReferenceError:“Vueisnotdefined“的原因与解决方案》在前端开发中,ReferenceError:Vueisnotdefined是一个常见... 目录一、错误描述二、错误成因分析三、解决方案1. 检查 vue.js 的引入方式2. 验证 npm 安装3.

PHP执行php.exe -v命令报错的解决方案

《PHP执行php.exe-v命令报错的解决方案》:本文主要介绍PHP执行php.exe-v命令报错的解决方案,文中通过图文讲解的非常详细,对大家的学习或工作有一定的帮助,需要的朋友可以参考下... 目录执行phpandroid.exe -v命令报错解决方案执行php.exe -v命令报错-PHP War

CentOS系统使用yum命令报错问题及解决

《CentOS系统使用yum命令报错问题及解决》文章主要讲述了在CentOS系统中使用yum命令时遇到的错误,并提供了个人解决方法,希望对大家有所帮助,并鼓励大家支持脚本之家... 目录Centos系统使用yum命令报错找到文件替换源文件为总结CentOS系统使用yum命令报错http://www.cppc

Linux 网络编程 --- 应用层

一、自定义协议和序列化反序列化 代码: 序列化反序列化实现网络版本计算器 二、HTTP协议 1、谈两个简单的预备知识 https://www.baidu.com/ --- 域名 --- 域名解析 --- IP地址 http的端口号为80端口,https的端口号为443 url为统一资源定位符。CSDNhttps://mp.csdn.net/mp_blog/creation/editor

客户案例:安全海外中继助力知名家电企业化解海外通邮困境

1、客户背景 广东格兰仕集团有限公司(以下简称“格兰仕”),成立于1978年,是中国家电行业的领军企业之一。作为全球最大的微波炉生产基地,格兰仕拥有多项国际领先的家电制造技术,连续多年位列中国家电出口前列。格兰仕不仅注重业务的全球拓展,更重视业务流程的高效与顺畅,以确保在国际舞台上的竞争力。 2、需求痛点 随着格兰仕全球化战略的深入实施,其海外业务快速增长,电子邮件成为了关键的沟通工具。

软考系统规划与管理师考试证书含金量高吗?

2024年软考系统规划与管理师考试报名时间节点: 报名时间:2024年上半年软考将于3月中旬陆续开始报名 考试时间:上半年5月25日到28日,下半年11月9日到12日 分数线:所有科目成绩均须达到45分以上(包括45分)方可通过考试 成绩查询:可在“中国计算机技术职业资格网”上查询软考成绩 出成绩时间:预计在11月左右 证书领取时间:一般在考试成绩公布后3~4个月,各地领取时间有所不同

ASIO网络调试助手之一:简介

多年前,写过几篇《Boost.Asio C++网络编程》的学习文章,一直没机会实践。最近项目中用到了Asio,于是抽空写了个网络调试助手。 开发环境: Win10 Qt5.12.6 + Asio(standalone) + spdlog 支持协议: UDP + TCP Client + TCP Server 独立的Asio(http://www.think-async.com)只包含了头文件,不依

安全管理体系化的智慧油站开源了。

AI视频监控平台简介 AI视频监控平台是一款功能强大且简单易用的实时算法视频监控系统。它的愿景是最底层打通各大芯片厂商相互间的壁垒,省去繁琐重复的适配流程,实现芯片、算法、应用的全流程组合,从而大大减少企业级应用约95%的开发成本。用户只需在界面上进行简单的操作,就可以实现全视频的接入及布控。摄像头管理模块用于多种终端设备、智能设备的接入及管理。平台支持包括摄像头等终端感知设备接入,为整个平台提

poj 3181 网络流,建图。

题意: 农夫约翰为他的牛准备了F种食物和D种饮料。 每头牛都有各自喜欢的食物和饮料,而每种食物和饮料都只能分配给一头牛。 问最多能有多少头牛可以同时得到喜欢的食物和饮料。 解析: 由于要同时得到喜欢的食物和饮料,所以网络流建图的时候要把牛拆点了。 如下建图: s -> 食物 -> 牛1 -> 牛2 -> 饮料 -> t 所以分配一下点: s  =  0, 牛1= 1~

2024网安周今日开幕,亚信安全亮相30城

2024年国家网络安全宣传周今天在广州拉开帷幕。今年网安周继续以“网络安全为人民,网络安全靠人民”为主题。2024年国家网络安全宣传周涵盖了1场开幕式、1场高峰论坛、5个重要活动、15场分论坛/座谈会/闭门会、6个主题日活动和网络安全“六进”活动。亚信安全出席2024年国家网络安全宣传周开幕式和主论坛,并将通过线下宣讲、创意科普、成果展示等多种形式,让广大民众看得懂、记得住安全知识,同时还