hutool、esayPoi、easyExcel、读写数据,性能对比

2024-03-01 10:36

本文主要是介绍hutool、esayPoi、easyExcel、读写数据,性能对比,希望对大家解决编程问题提供一定的参考价值,需要的开发者们随着小编来一起学习吧!

目录

1.源码下载 

2、先上结论

3、代码

3.1、实体、接口

3.2、读CSV

3.3、读Excel

3.4、写CSV

3.5、写Excel

3.6、业务示例


1.源码下载 

示例源码下载(含模板)

2、先上结论

注意注意::::导出大文件的时候可能会有临时存储,
我的是在C:\Users\xbx\AppData\Local\Temp\poifiles读csv文件
行数:1048575
hutool:      5205
esayPoi:     3929
easyExcel1:  1917
easyExcel2:  2832读excel文件
行数:1048575
hutool:     58207
esayPoi:    25513
easyExce1:  10646
easyExcel2: 10123写csv文件:
行数:1048575
hutool:     6136
esayPoi:     828
easyExcel1: 6943
easyExcel2: 5212写excel文件:
行数:1048575
hutool1:  146655
hutool2:   23570
esayPoi1:  21805
esayPoi2:  93034
easyExcel1:20762
easyExcel2:20523
easyExcel3:15386
easyExcel4:14586

3、代码

3.1、实体、接口

package cn.sdjtky.ltpon.FileTest;import cn.afterturn.easypoi.excel.annotation.Excel;
import lombok.Data;/*** @version v1.0* @ProjectName: ltpon* @ClassName: Strainentey* @Description:* @Author: xbx* @Date: 2023/12/8 13:55*/
@Data
public class DeviceDto {//sn+channelString sn;//与地表距离(m)Double surfaceDistance;//结构层位String structuralLayer;//就写电信号String channel;//这个值就是StressStrainDataDto里面的key 如:"t.v1","t.v2"String dataKey;
}
package cn.sdjtky.ltpon.FileTest;import cn.hutool.core.annotation.Alias;
import lombok.Data;/*** @version v1.0* @ProjectName: ltpon* @ClassName: Strainentey* @Description:* @Author: xbx* @Date: 2023/12/8 13:55*/
@Data
public class StrainEntity {@Alias("时间")String time;@Alias("通道1")String v1;@Alias("通道2")String v2;@Alias("通道3")String v3;@Alias("通道4")String v4;Double v5;Double v6;Double v7;Double v8;
}
package cn.sdjtky.ltpon.FileTest;import cn.afterturn.easypoi.excel.annotation.Excel;
import lombok.Data;/*** @version v1.0* @ProjectName: ltpon* @ClassName: Strainentey* @Description:* @Author: xbx* @Date: 2023/12/8 13:55*/
@Data
public class StrainEntity2 {@Excel(name = "时间")String time;@Excel(name = "通道1")String v1;@Excel(name = "通道2")String v2;@Excel(name = "通道3")String v3;@Excel(name = "通道4")String v4;Double v5;Double v6;Double v7;Double v8;
}
package cn.sdjtky.ltpon.FileTest;import com.alibaba.excel.annotation.ExcelProperty;
import lombok.Data;/*** @version v1.0* @ProjectName: ltpon* @ClassName: Strainentey* @Description:* @Author: xbx* @Date: 2023/12/8 13:55*/
@Data
public class StrainEntity3 {@ExcelProperty("时间")String time;@ExcelProperty("通道1")String v1;@ExcelProperty("通道2")String v2;@ExcelProperty("通道3")String v3;@ExcelProperty("通道4")String v4;Double v5;Double v6;Double v7;Double v8;
}
package cn.sdjtky.ltpon.FileTest;import cn.afterturn.easypoi.excel.annotation.Excel;
import lombok.Data;import java.util.Date;/*** @version v1.0* @ProjectName: ltpon* @ClassName: Strainentey* @Description:* @Author: xbx* @Date: 2023/12/8 13:55*/
@Data
public class StressStrainDataDto {
//    年/月/日String date;
//    时/分/秒/毫秒String time;
//    时/分/秒/毫秒String dateTime;Double v1;Double v2;Double v3;Double v4;Double v5;Double v6;Double v7;Double v8;Double v9;Double v10;Double v11;Double v12;Double v13;
}
package cn.sdjtky.ltpon.FileTest;import cn.sdjtky.ltpon.FileTest.StrainEntity3;
import com.alibaba.excel.context.AnalysisContext;
import com.alibaba.excel.read.listener.ReadListener;
import com.alibaba.excel.util.ListUtils;
import com.alibaba.fastjson2.JSON;
import org.apache.poi.ss.formula.functions.T;import java.util.ArrayList;
import java.util.List;// 有个很重要的点 DemoDataListener 不能被spring管理,要每次读取excel都要new,然后里面用到spring可以构造方法传进去
public class Strain3ReadListener<T> implements ReadListener<T> {/*** 每隔5条存储数据库,实际使用中可以100条,然后清理list ,方便内存回收*//*** 缓存的数据*/private List<T> cachedDataList = new ArrayList<T>();/*** 这个每一条数据解析都会来调用** @param data    one row value. Is is same as {@link AnalysisContext#readRowHolder()}* @param context*/@Overridepublic void invoke(T data, AnalysisContext context) {cachedDataList.add(data);
//        // 达到BATCH_COUNT了,需要去存储一次数据库,防止数据几万条数据在内存,容易OOM
//        if (cachedDataList.size() >= BATCH_COUNT) {
//            // 存储完成清理 list
//            cachedDataList = ListUtils.newArrayListWithExpectedSize(BATCH_COUNT);
//        }}/*** 所有数据解析完成了 都会来调用** @param context*/@Overridepublic void doAfterAllAnalysed(AnalysisContext context) {// 这里也要保存数据,确保最后遗留的数据也存储到数据库}public List<T> getCachedDataList() {return cachedDataList;}
}

3.2、读CSV

package cn.sdjtky.ltpon.FileTest;import cn.afterturn.easypoi.csv.CsvImportUtil;
import cn.afterturn.easypoi.csv.entity.CsvImportParams;
import cn.afterturn.easypoi.excel.entity.ImportParams;
import cn.hutool.core.io.resource.ResourceUtil;
import cn.hutool.core.text.csv.CsvReader;
import cn.hutool.core.text.csv.CsvUtil;
import com.alibaba.excel.EasyExcel;
import com.alibaba.excel.context.AnalysisContext;
import com.alibaba.excel.event.AnalysisEventListener;
import com.alibaba.excel.support.ExcelTypeEnum;
import org.junit.jupiter.api.Test;
import org.slf4j.Logger;
import org.slf4j.LoggerFactory;import java.io.File;
import java.io.FileInputStream;
import java.util.ArrayList;
import java.util.List;public class ReadCsvTest {private static final Logger LOGGER = LoggerFactory.getLogger(ReadCsvTest.class);@Testpublic void test() {String filePath = "G:/test/1048575.csv";hutool(filePath);
//        esayPoi(filePath);
//        easyExcel1(filePath);
//        easyExcel2(filePath);}public void hutool(String filePath) {try {long start = System.currentTimeMillis();CsvReader reader = CsvUtil.getReader();List<StrainEntity> result = reader.read(ResourceUtil.getUtf8Reader(filePath), StrainEntity.class);long end = System.currentTimeMillis();System.out.println("hutool:"+ (end-start));System.out.println("行数:"+result.size());} catch (Exception e) {e.printStackTrace();}}public void esayPoi(String filePath) {try {CsvImportParams params = new CsvImportParams(CsvImportParams.UTF8);long start = System.currentTimeMillis();List<StrainEntity2> result = CsvImportUtil.importCsv(new FileInputStream(new File(filePath)),StrainEntity2.class,params);long end = System.currentTimeMillis();System.out.println("esayPoi:"+ (end-start));System.out.println("行数:"+result.size());} catch (Exception e) {e.printStackTrace();}}public void easyExcel1(String filePath) {try {ImportParams params = new ImportParams();List<StrainEntity3> result = new ArrayList<StrainEntity3>();long start = System.currentTimeMillis();EasyExcel.read(filePath, StrainEntity3.class, new AnalysisEventListener<StrainEntity3>() {@Overridepublic void invoke(StrainEntity3 reportExcel, AnalysisContext analysisContext) {// 将读取到的每一行存入reportDetails集合中result.add(reportExcel);}@Overridepublic void doAfterAllAnalysed(AnalysisContext analysisContext) {}}).excelType(ExcelTypeEnum.CSV).sheet().doRead();long end = System.currentTimeMillis();System.out.println("easyExcel1:"+ (end-start));System.out.println("行数:"+result.size());} catch (Exception e) {e.printStackTrace();}}public void easyExcel2(String filePath) {final Strain3ReadListener<StrainEntity3> strain3ReadListener = new Strain3ReadListener<StrainEntity3>() ;try {long start = System.currentTimeMillis();EasyExcel.read(filePath, StrainEntity3.class, strain3ReadListener).sheet().doRead();
//            EasyExcel.read(filePath)
//                    .excelType(ExcelTypeEnum.CSV)
//                    .registerReadListener(strain3ReadListener)
//                    .sheet()
//                    .doRead();List<StrainEntity3> result = strain3ReadListener.getCachedDataList();long end = System.currentTimeMillis();System.out.println("easyExcel2:"+ (end-start));System.out.println("行数:"+result.size());} catch (Exception e) {e.printStackTrace();}}}

3.3、读Excel

package cn.sdjtky.ltpon.FileTest;import cn.afterturn.easypoi.excel.ExcelImportUtil;
import cn.afterturn.easypoi.excel.entity.ImportParams;
import cn.afterturn.easypoi.handler.inter.IReadHandler;
import cn.hutool.poi.excel.ExcelReader;
import cn.hutool.poi.excel.ExcelUtil;
import cn.sdjtky.ltpon.easy.excel.listener.MergeCellsReadListener;
import cn.sdjtky.ltpon.poi.DemoDataListener;
import com.alibaba.excel.EasyExcel;
import com.alibaba.excel.context.AnalysisContext;
import com.alibaba.excel.enums.CellExtraTypeEnum;
import com.alibaba.excel.event.AnalysisEventListener;
import com.alibaba.excel.read.listener.PageReadListener;
import com.alibaba.excel.support.ExcelTypeEnum;
import com.alibaba.fastjson2.JSON;
import org.junit.jupiter.api.Test;
import org.slf4j.Logger;
import org.slf4j.LoggerFactory;import java.io.File;
import java.io.FileInputStream;
import java.util.ArrayList;
import java.util.List;public class ReadExcelTest {private static final Logger LOGGER = LoggerFactory.getLogger(ReadExcelTest.class);@Testpublic void test() {String filePath = "G:/test/1048575.xlsx";hutool(filePath);
//        esayPoi(filePath);
//        easyExcel1(filePath);
//        easyExcel2(filePath);}public void hutool(String filePath) {try {long start = System.currentTimeMillis();ExcelReader reader = ExcelUtil.getReader(filePath);List<StrainEntity> result = reader.readAll(StrainEntity.class);long end = System.currentTimeMillis();System.out.println("hutool:"+ (end-start));System.out.println("行数:"+result.size());} catch (Exception e) {e.printStackTrace();}}public void esayPoi(String filePath) {try {ImportParams params = new ImportParams();List<StrainEntity2> result = new ArrayList<StrainEntity2>();long start = System.currentTimeMillis();ExcelImportUtil.importExcelBySax(new FileInputStream(new File(filePath)),StrainEntity2.class,params,new IReadHandler<StrainEntity2>() {@Overridepublic void handler(StrainEntity2 o) {result.add(o);}@Overridepublic void doAfterAll() {System.out.println("全部执行完毕了--------------------------------");}});long end = System.currentTimeMillis();System.out.println("esayPoi:"+ (end-start));System.out.println("行数:"+result.size());} catch (Exception e) {e.printStackTrace();}}public void easyExcel1(String filePath) {try {ImportParams params = new ImportParams();List<StrainEntity3> result = new ArrayList<StrainEntity3>();long start = System.currentTimeMillis();EasyExcel.read(filePath, StrainEntity3.class, new AnalysisEventListener<StrainEntity3>() {@Overridepublic void invoke(StrainEntity3 reportExcel, AnalysisContext analysisContext) {// 将读取到的每一行存入reportDetails集合中result.add(reportExcel);}@Overridepublic void doAfterAllAnalysed(AnalysisContext analysisContext) {}}).sheet().doRead();long end = System.currentTimeMillis();System.out.println("easyExcel1:"+ (end-start));System.out.println("行数:"+result.size());} catch (Exception e) {e.printStackTrace();}}public void easyExcel2(String filePath) {final Strain3ReadListener<StrainEntity3> strain3ReadListener = new Strain3ReadListener<StrainEntity3>() ;try {long start = System.currentTimeMillis();EasyExcel.read(filePath, StrainEntity3.class, strain3ReadListener).sheet().doRead();
//            EasyExcel.read(filePath)
//                    .registerReadListener(strain3ReadListener)
//                    .sheet()
//                    .doRead();List<StrainEntity3> result = strain3ReadListener.getCachedDataList();long end = System.currentTimeMillis();System.out.println("easyExcel2:"+ (end-start));System.out.println("行数:"+result.size());} catch (Exception e) {e.printStackTrace();}}}

3.4、写CSV

package cn.sdjtky.ltpon.FileTest;import cn.afterturn.easypoi.csv.CsvExportUtil;
import cn.afterturn.easypoi.csv.CsvImportUtil;
import cn.afterturn.easypoi.csv.entity.CsvExportParams;
import cn.afterturn.easypoi.csv.entity.CsvImportParams;
import cn.afterturn.easypoi.excel.entity.ImportParams;
import cn.hutool.core.io.FastByteArrayOutputStream;
import cn.hutool.core.io.resource.ResourceUtil;
import cn.hutool.core.text.csv.CsvReader;
import cn.hutool.core.text.csv.CsvUtil;
import cn.hutool.core.text.csv.CsvWriter;
import cn.hutool.core.util.CharsetUtil;
import cn.sdjtky.ltpon.query.HumidityQuery;
import cn.sdjtky.ltpon.utils.BeanCopierUtils;
import com.alibaba.excel.EasyExcel;
import com.alibaba.excel.ExcelWriter;
import com.alibaba.excel.context.AnalysisContext;
import com.alibaba.excel.event.AnalysisEventListener;
import com.alibaba.excel.support.ExcelTypeEnum;
import com.alibaba.excel.write.metadata.WriteSheet;
import com.ruoyi.common.utils.uuid.IdUtils;
import org.junit.jupiter.api.Test;
import org.slf4j.Logger;
import org.slf4j.LoggerFactory;
import org.springframework.beans.BeanUtils;import java.io.ByteArrayInputStream;
import java.io.File;
import java.io.FileInputStream;
import java.io.FileOutputStream;
import java.util.ArrayList;
import java.util.List;public class WriteCsvTest {private static final Logger LOGGER = LoggerFactory.getLogger(WriteCsvTest.class);@Testpublic void test() {String filePath = "G:/test/1048575.csv";List<StrainEntity3> strainEntity3List  = Rerad_easyExcel1(filePath);List<StrainEntity> strainEntityList = BeanCopierUtils.list(strainEntity3List, StrainEntity::new);List<StrainEntity2> strainEntity2List = BeanCopierUtils.list(strainEntity3List, StrainEntity2::new);
//        List<StrainEntity3> strainEntity3List = BeanCopierUtils.list(strainEntity3List, StrainEntity3::new);
//        hutool(strainEntityList);
//        esayPoi(strainEntity2List);
//        easyExcel1(strainEntity3List);easyExcel2(strainEntity3List);}public List<StrainEntity3> Rerad_easyExcel1(String filePath) {List<StrainEntity3> result = new ArrayList<StrainEntity3>();try {ImportParams params = new ImportParams();EasyExcel.read(filePath, StrainEntity3.class, new AnalysisEventListener<StrainEntity3>() {@Overridepublic void invoke(StrainEntity3 reportExcel, AnalysisContext analysisContext) {// 将读取到的每一行存入reportDetails集合中result.add(reportExcel);}@Overridepublic void doAfterAllAnalysed(AnalysisContext analysisContext) {}}).excelType(ExcelTypeEnum.CSV).sheet().doRead();} catch (Exception e) {e.printStackTrace();}return result;}public void hutool(List<StrainEntity> list) {System.out.println("数据行数:"+list.size());try {long start = System.currentTimeMillis();CsvWriter writer = CsvUtil.getWriter("G:/test/write/hutool.csv", CharsetUtil.CHARSET_UTF_8);
//            writer.write(list);writer.writeBeans(list);long end = System.currentTimeMillis();long cost = end-start;System.out.println("hutool:"+ cost);} catch (Exception e) {e.printStackTrace();}}public void esayPoi(List<StrainEntity2> list) {System.out.println("数据行数:"+list.size());try {long start = System.currentTimeMillis();CsvExportParams params = new CsvExportParams(CsvImportParams.UTF8);FileOutputStream fos = new FileOutputStream("G:/test/write/esayPoi.csv");CsvExportUtil.exportCsv(params, StrainEntity2.class, list, fos);fos.flush();fos.close();long end = System.currentTimeMillis();long cost = end-start;System.out.println("esayPoi:"+ cost);} catch (Exception e) {e.printStackTrace();}}public void easyExcel1(List<StrainEntity3> list) {System.out.println("数据行数:"+list.size());try {long start = System.currentTimeMillis();FastByteArrayOutputStream fastByteArrayOutputStream = new FastByteArrayOutputStream();try {EasyExcel.write(fastByteArrayOutputStream, StrainEntity3.class).autoCloseStream(false).excelType(ExcelTypeEnum.CSV).sheet("sheet页").doWrite(list);FileOutputStream fileOutputStream = new FileOutputStream("G:/test/write/easyExcel.csv");fastByteArrayOutputStream.writeTo(fileOutputStream);fileOutputStream.flush();fileOutputStream.close();} finally {}long end = System.currentTimeMillis();long cost = end-start;System.out.println("easyExcel1:"+ cost);} catch (Exception e) {e.printStackTrace();}}public void easyExcel2(List<StrainEntity3> list) {System.out.println("数据行数:"+list.size());try {long start = System.currentTimeMillis();try {EasyExcel.write("G:/test/write/easyExcel.csv", StrainEntity3.class).excelType(ExcelTypeEnum.CSV).sheet("sss").doWrite(list);} finally {}long end = System.currentTimeMillis();long cost = end-start;System.out.println("easyExcel2:"+ cost);} catch (Exception e) {e.printStackTrace();}}}

3.5、写Excel

package cn.sdjtky.ltpon.FileTest;import cn.afterturn.easypoi.excel.ExcelExportUtil;
import cn.afterturn.easypoi.excel.ExcelImportUtil;
import cn.afterturn.easypoi.excel.entity.ExportParams;
import cn.afterturn.easypoi.excel.entity.ImportParams;
import cn.afterturn.easypoi.excel.entity.TemplateExportParams;
import cn.afterturn.easypoi.handler.inter.IExcelExportServer;
import cn.afterturn.easypoi.handler.inter.IReadHandler;
import cn.hutool.core.bean.BeanUtil;
import cn.hutool.core.collection.CollUtil;
import cn.hutool.core.io.FastByteArrayOutputStream;
import cn.hutool.core.text.csv.CsvUtil;
import cn.hutool.core.text.csv.CsvWriter;
import cn.hutool.core.util.CharsetUtil;
import cn.hutool.poi.excel.BigExcelWriter;
import cn.hutool.poi.excel.ExcelReader;
import cn.hutool.poi.excel.ExcelUtil;
import cn.hutool.poi.excel.ExcelWriter;
import cn.sdjtky.ltpon.utils.BeanCopierUtils;
import com.alibaba.excel.EasyExcel;
import com.alibaba.excel.context.AnalysisContext;
import com.alibaba.excel.event.AnalysisEventListener;
import com.alibaba.excel.support.ExcelTypeEnum;
import com.alibaba.excel.write.metadata.WriteSheet;
import com.alibaba.excel.write.metadata.fill.FillWrapper;
import com.ruoyi.common.utils.uuid.IdUtils;
import lombok.SneakyThrows;
import org.apache.poi.ss.usermodel.Workbook;
import org.apache.poi.xssf.streaming.SXSSFWorkbook;
import org.junit.jupiter.api.Test;
import org.slf4j.Logger;
import org.slf4j.LoggerFactory;import java.io.File;
import java.io.FileInputStream;
import java.io.FileOutputStream;
import java.util.ArrayList;
import java.util.HashMap;
import java.util.List;
import java.util.Map;public class WriteExcelTest {private static final Logger LOGGER = LoggerFactory.getLogger(WriteExcelTest.class);@Testpublic void test() {String filePath = "G:/test/1048575.csv";List<StrainEntity3> strainEntity3List  = Rerad_easyExcel1(filePath);List<StrainEntity> strainEntityList = BeanCopierUtils.list(strainEntity3List, StrainEntity::new);List<StrainEntity2> strainEntity2List = BeanCopierUtils.list(strainEntity3List, StrainEntity2::new);
//        hutool1(strainEntityList);
//        hutool2(strainEntityList);
//        esayPoi1(strainEntity2List);
//        esayPoi2(strainEntity2List);
//        easyExcel1(strainEntity3List);
//        easyExcel2(strainEntity3List);
//        easyExcel3(strainEntity3List);easyExcel4(strainEntity3List);}public List<StrainEntity3> Rerad_easyExcel1(String filePath) {List<StrainEntity3> result = new ArrayList<StrainEntity3>();try {ImportParams params = new ImportParams();EasyExcel.read(filePath, StrainEntity3.class, new AnalysisEventListener<StrainEntity3>() {@Overridepublic void invoke(StrainEntity3 reportExcel, AnalysisContext analysisContext) {// 将读取到的每一行存入reportDetails集合中result.add(reportExcel);}@Overridepublic void doAfterAllAnalysed(AnalysisContext analysisContext) {}}).excelType(ExcelTypeEnum.CSV).sheet().doRead();} catch (Exception e) {e.printStackTrace();}return result;}public void hutool1(List<StrainEntity> list) {System.out.println("数据行数:"+list.size());try {long start = System.currentTimeMillis();ExcelWriter writer = ExcelUtil.getWriter("G:/test/write/hutool.xlsx");writer.write(list, true);// 关闭writer,释放内存writer.close();long end = System.currentTimeMillis();long cost = end-start;System.out.println("hutool:"+ cost);} catch (Exception e) {e.printStackTrace();}}public void hutool2(List<StrainEntity> list) {System.out.println("数据行数:"+list.size());try {long start = System.currentTimeMillis();BigExcelWriter writer= ExcelUtil.getBigWriter("G:/test/write/hutool2"+ IdUtils.getSequenceID()+".xlsx");writer.write(list);writer.close();long end = System.currentTimeMillis();long cost = end-start;System.out.println("hutool2:"+ cost);} catch (Exception e) {e.printStackTrace();}}public void esayPoi1(List<StrainEntity2> list) {System.out.println("数据行数:"+list.size());try {long start = System.currentTimeMillis();ExportParams params   = new ExportParams();params.setMaxNum(1048576);Workbook  workbook = null;workbook = ExcelExportUtil.exportBigExcel(params, StrainEntity2.class, new IExcelExportServer() {@Overridepublic List<Object> selectListForExcelExport(Object obj, int page) {if (list.size()<=0){return null;}List<Object> listObj = new ArrayList<Object>();listObj.addAll(list);list.clear();return listObj;}}, 1);//           workbook = ExcelExportUtil.exportExcel(params,StrainEntity2.class,list);FileOutputStream fos = new FileOutputStream("G:/test/write/esayPoi.xlsx");workbook.write(fos);fos.close();long end = System.currentTimeMillis();long cost = end-start;System.out.println("esayPoi:"+ cost);} catch (Exception e) {e.printStackTrace();}}public void esayPoi2(List<StrainEntity2> list) {System.out.println("数据行数:"+list.size());try {long start = System.currentTimeMillis();Map<String, Object> map = new HashMap<String, Object>();map.put("list", list);TemplateExportParams params = new TemplateExportParams("G:/test/t/StrainEntity2.xlsx");Workbook workbook = ExcelExportUtil.exportExcel(params, map);FileOutputStream fos = new FileOutputStream("G:/test/write/esayPoi2.xlsx");workbook.write(fos);fos.close();long end = System.currentTimeMillis();long cost = end-start;System.out.println("esayPoi2:"+ cost);} catch (Exception e) {e.printStackTrace();}}public void easyExcel1(List<StrainEntity3> list) {System.out.println("数据行数:"+list.size());try {long start = System.currentTimeMillis();FastByteArrayOutputStream fastByteArrayOutputStream = new FastByteArrayOutputStream();try {EasyExcel.write(fastByteArrayOutputStream, StrainEntity3.class).autoCloseStream(false).excelType(ExcelTypeEnum.XLSX).sheet("sheet页").doWrite(list);FileOutputStream fileOutputStream = new FileOutputStream("G:/test/write/easyExcel.xlsx");fastByteArrayOutputStream.writeTo(fileOutputStream);fileOutputStream.flush();fileOutputStream.close();} finally {}long end = System.currentTimeMillis();long cost = end-start;System.out.println("easyExcel1:"+ cost);} catch (Exception e) {e.printStackTrace();}}public void easyExcel2(List<StrainEntity3> list) {System.out.println("数据行数:"+list.size());try {long start = System.currentTimeMillis();try {EasyExcel.write("G:/test/write/easyExcel.xlsx", StrainEntity3.class).excelType(ExcelTypeEnum.XLSX).sheet("sss").doWrite(list);} finally {}long end = System.currentTimeMillis();long cost = end-start;System.out.println("easyExcel2:"+ cost);} catch (Exception e) {e.printStackTrace();}}public void easyExcel3(List<StrainEntity3> list) {System.out.println("数据行数:"+list.size());try {long start = System.currentTimeMillis();String templateFileName = "G:/test/t/StrainEntity3.xlsx";String fileName = "G:/test/write/easyExcel.xlsx";// 方案1 一下子全部放到内存里面 并填充// 这里 会填充到第一个sheet, 然后文件流会自动关闭EasyExcel.write(fileName).withTemplate(templateFileName).sheet().doFill(list);long end = System.currentTimeMillis();long cost = end-start;System.out.println("easyExcel3:"+ cost);} catch (Exception e) {e.printStackTrace();}}public void easyExcel4(List<StrainEntity3> list) {System.out.println("数据行数:"+list.size());try {long start = System.currentTimeMillis();FastByteArrayOutputStream fastByteArrayOutputStream = new FastByteArrayOutputStream();// 方案2 分多次 填充 会使用文件缓存(省内存)String templateFileName = "G:/test/t/StrainEntity3.xlsx";String fileName = "G:/test/write/easyExcel.xlsx";try (com.alibaba.excel.ExcelWriter excelWriter = EasyExcel.write(fastByteArrayOutputStream).withTemplate(templateFileName).build()) {WriteSheet writeSheet = EasyExcel.writerSheet().build();excelWriter.fill(list, writeSheet);}//            String templateFileName = "G:/test/t/StrainEntity3.1.xlsx";
//            String fileName = "G:/test/write/easyExcel.xlsx";
//            try (com.alibaba.excel.ExcelWriter excelWriter = EasyExcel.write(fileName).withTemplate(templateFileName).build()) {
//                WriteSheet writeSheet = EasyExcel.writerSheet().build();
//                excelWriter.fill(new FillWrapper("sl", list),  writeSheet);
//            }//            String templateFileName = "G:/test/t/StrainEntity3.1.xlsx";
//            String fileName = "G:/test/write/easyExcel.xlsx";
//            try (com.alibaba.excel.ExcelWriter excelWriter = EasyExcel.write(fileName).withTemplate(templateFileName).build()) {
//                WriteSheet writeSheet = EasyExcel.writerSheet().build();
//                excelWriter.fill(new FillWrapper("sl", listX),  writeSheet);
//                excelWriter.fill(new FillWrapper("sl", listX),  writeSheet);
//                excelWriter.fill(new FillWrapper("sl", listX),  writeSheet);
//                excelWriter.fill(new FillWrapper("sl", listX),  writeSheet);
//                excelWriter.fill(new FillWrapper("sl", listX),  writeSheet);
//            }long end = System.currentTimeMillis();long cost = end-start;System.out.println("easyExcel4:"+ cost);} catch (Exception e) {e.printStackTrace();}}
}

3.6、业务示例

package cn.sdjtky.ltpon.FileTest;import cn.afterturn.easypoi.excel.ExcelExportUtil;
import cn.afterturn.easypoi.excel.entity.ExportParams;
import cn.afterturn.easypoi.excel.entity.ImportParams;
import cn.afterturn.easypoi.excel.entity.TemplateExportParams;
import cn.afterturn.easypoi.handler.inter.IExcelExportServer;
import cn.hutool.core.io.FastByteArrayOutputStream;
import cn.hutool.poi.excel.BigExcelWriter;
import cn.hutool.poi.excel.ExcelUtil;
import cn.hutool.poi.excel.ExcelWriter;
import cn.sdjtky.ltpon.utils.BeanCopierUtils;
import cn.sdjtky.ltpon.utils.LocalDateTimeUtils;
import com.alibaba.excel.EasyExcel;
import com.alibaba.excel.context.AnalysisContext;
import com.alibaba.excel.enums.WriteDirectionEnum;
import com.alibaba.excel.event.AnalysisEventListener;
import com.alibaba.excel.support.ExcelTypeEnum;
import com.alibaba.excel.write.metadata.WriteSheet;
import com.alibaba.excel.write.metadata.fill.FillConfig;
import com.alibaba.excel.write.metadata.fill.FillWrapper;
import com.ruoyi.common.utils.DateUtils;
import com.ruoyi.common.utils.uuid.IdUtils;
import org.apache.poi.ss.usermodel.Workbook;
import org.junit.jupiter.api.Test;
import org.slf4j.Logger;
import org.slf4j.LoggerFactory;
import org.springframework.core.io.DefaultResourceLoader;
import org.springframework.core.io.Resource;import java.io.ByteArrayOutputStream;
import java.io.FileOutputStream;
import java.time.LocalDateTime;
import java.util.*;public class StressStrainTest {private static final Logger LOGGER = LoggerFactory.getLogger(StressStrainTest.class);private static final String stationName = "临沂G205观测点";private static final String pileNumber = "K103--K113";private static final String observer = "徐本锡";private static final String observationDate = LocalDateTimeUtils.format(LocalDateTimeUtils.DEFAULT_PATTERN,LocalDateTime.now());private static final String reviewer = "金小少";private static final String reviewDate = LocalDateTimeUtils.format(LocalDateTimeUtils.DEFAULT_PATTERN,LocalDateTime.now());@Testpublic void test() {
//        esayPoi2();easyExcel4();
//        easyExcelCSV2();}public Map<String, Object> getData(){Map<String, Object> map = new HashMap<String, Object>();List<DeviceDto> deviceList = new ArrayList<DeviceDto>();for (int i=1; i<=8; i++){DeviceDto deviceDto= new DeviceDto();deviceDto.setSn("传感器x_"+i);deviceDto.setSurfaceDistance(Double.valueOf(i));deviceDto.setStructuralLayer("层位"+i);deviceDto.setChannel("电信号");deviceDto.setDataKey("t.v"+i);deviceList.add(deviceDto);}List<Map<String, Object>> dataList = new ArrayList<Map<String, Object>>();List<StressStrainDataDto> dataList2 = new ArrayList<StressStrainDataDto>();for(int i=0; i<100; i++){Map<String, Object> m = new HashMap<String, Object>();StressStrainDataDto dto = new StressStrainDataDto();for (int j=1; j<=8; j++){m.put("date", DateUtils.datePath());m.put("time", DateUtils.dateTimeNow(DateUtils.HHMMssSSS_1));m.put("v"+j, Math.random()+1);dto.setDate(DateUtils.datePath());dto.setTime(DateUtils.dateTimeNow(DateUtils.HHMMssSSS_1));dto.setDateTime(DateUtils.dateTimeNow(DateUtils.YYYYMMDDHHMMssSSS_1));dto.setV1( Math.random()+1);dto.setV2( Math.random()+1);dto.setV3( Math.random()+1);dto.setV4( Math.random()+1);dto.setV5( Math.random()+1);dto.setV6( Math.random()+1);dto.setV7( Math.random()+1);dto.setV8( Math.random()+1);}dataList.add(m);dataList2.add(dto);}map.put("stationName", stationName);map.put("pileNumber", pileNumber);map.put("observer", observer);map.put("observationDate", observationDate);map.put("reviewer", reviewer);map.put("reviewDate", reviewDate);map.put("deviceList", deviceList);map.put("dataList", dataList2);return map;}public void esayPoi2() {Map<String, Object> data = getData();System.out.println("数据行数:"+((List)data.get("dataList")).size());try {long start = System.currentTimeMillis();TemplateExportParams params = new TemplateExportParams("G:/test/t/stressStrainTemplateDongHua.xlsx");params.setColForEach(true);Workbook workbook = ExcelExportUtil.exportExcel(params, data);FileOutputStream fos = new FileOutputStream("G:/test/write/stressStrainTemplateDongHua.xlsx");workbook.write(fos);fos.close();long end = System.currentTimeMillis();long cost = end-start;System.out.println("esayPoi2:"+ cost);} catch (Exception e) {e.printStackTrace();}}public void easyExcel4() {Map<String, Object> data = getData();Map<String, Object> data1 = new HashMap<String, Object>();data1.put("stationName", data.get("stationName"));data1.put("pileNumber", data.get("pileNumber"));data1.put("observer", data.get("observer"));data1.put("observationDate", data.get("observationDate"));data1.put("reviewer", data.get("reviewer"));data1.put("reviewDate", data.get("reviewDate"));List<DeviceDto> deviceList = (List<DeviceDto>) data.get("deviceList");List<StressStrainDataDto> dataList = (List<StressStrainDataDto>) data.get("dataList");// 根据用户传入字段 导出数据Set<String> includeColumnFiledNames = new HashSet<String>();includeColumnFiledNames.add("date");
//        includeColumnFiledNames.add("time");for(int i=0; i<deviceList.size(); i++){includeColumnFiledNames.add("v"+(i+1));}System.out.println("数据行数:"+((List)data.get("dataList")).size());try {long start = System.currentTimeMillis();Resource resource = new DefaultResourceLoader().getResource("classpath:templates/stressStrain/stressStrainTemplateDongHua.xlsx");FastByteArrayOutputStream outputStream = new FastByteArrayOutputStream();// 方案2 分多次 填充 会使用文件缓存(省内存)
//            String templateFileName = "G:/test/t/stressStrainTemplateDongHua2.xlsx";
//            try (com.alibaba.excel.ExcelWriter excelWriter = EasyExcel.write(fastByteArrayOutputStream).withTemplate(templateFileName).build()) {
//                WriteSheet writeSheet = EasyExcel.writerSheet().build();
//                FillConfig fillConfig = FillConfig.builder().autoStyle(true).direction(WriteDirectionEnum.HORIZONTAL).build();
//                excelWriter.fill(data1,  writeSheet);
//                excelWriter.fill(new FillWrapper("deviceList", deviceList), fillConfig, writeSheet);writeSheet.setIncludeColumnFieldNames(includeColumnFiledNames);
//                excelWriter.fill(new FillWrapper("dataList", dataList),  writeSheet);
//            }try (com.alibaba.excel.ExcelWriter excelWriter = EasyExcel.write(outputStream).withTemplate(resource.getInputStream()).build()) {WriteSheet writeSheet = EasyExcel.writerSheet().build();FillConfig fillConfig = FillConfig.builder().autoStyle(true).direction(WriteDirectionEnum.HORIZONTAL).build();excelWriter.fill(data1,  writeSheet);excelWriter.fill(new FillWrapper("deviceList", deviceList), fillConfig, writeSheet);excelWriter.fill(new FillWrapper("dataList", dataList), writeSheet);excelWriter.close();outputStream.flush();outputStream.close();}
//            FileOutputStream fileOutputStream = new FileOutputStream("G:/test/write/123.xlsx");
//            outputStream.writeTo(fileOutputStream);
//            fileOutputStream.flush();
//            fileOutputStream.close();long end = System.currentTimeMillis();
//            String templateFileName = "G:/test/t/stressStrainTemplateDongHua2.xlsx";
//            String fileName = "G:/test/write/stressStrainTemplateDongHua2.xlsx";
//            try (com.alibaba.excel.ExcelWriter excelWriter = EasyExcel.write(fileName).withTemplate(resource.getInputStream()).build()) {
//                WriteSheet writeSheet = EasyExcel.writerSheet().build();
//                FillConfig fillConfig = FillConfig.builder().autoStyle(true).direction(WriteDirectionEnum.HORIZONTAL).build();
//                excelWriter.fill(data1,  writeSheet);
//                excelWriter.fill(new FillWrapper("deviceList", deviceList), fillConfig, writeSheet);writeSheet.setIncludeColumnFieldNames(includeColumnFiledNames);
//                excelWriter.fill(new FillWrapper("dataList", dataList),  writeSheet);
//            }long cost = end-start;System.out.println("easyExcel4:"+ cost);} catch (Exception e) {e.printStackTrace();}}public void easyExcelCSV2() {Map<String, Object> data = getData();List<DeviceDto> deviceList = (List<DeviceDto>) data.get("deviceList");List<StressStrainDataDto> dataList = (List<StressStrainDataDto>) data.get("dataList");// 根据用户传入字段 导出数据Set<String> includeColumnFiledNames = new HashSet<String>();List<List<String>> headList = new ArrayList<List<String>>();List<String> head0 = new ArrayList<String>();head0.add("时间");head0.add("与路表距离");head0.add("结构层位");headList.add(head0);includeColumnFiledNames.add("dateTime");for(int i=0; i<deviceList.size(); i++){includeColumnFiledNames.add("v"+(i+1));List<String> head = new ArrayList<String>();head.add(deviceList.get(i).getSn());head.add(deviceList.get(i).getSurfaceDistance().toString());head.add(deviceList.get(i).getStructuralLayer());headList.add(head);}System.out.println("数据行数:"+((List)data.get("dataList")).size());try {long start = System.currentTimeMillis();try {EasyExcel.write("G:/test/write/stressStrainTemplateDongHua2.csv", StressStrainDataDto.class).head(headList).includeColumnFieldNames(includeColumnFiledNames).excelType(ExcelTypeEnum.CSV).sheet("sss").doWrite(dataList);} finally {}long end = System.currentTimeMillis();long cost = end-start;System.out.println("easyExcel2:"+ cost);} catch (Exception e) {e.printStackTrace();}}}

这篇关于hutool、esayPoi、easyExcel、读写数据,性能对比的文章就介绍到这儿,希望我们推荐的文章对编程师们有所帮助!



http://www.chinasem.cn/article/762148

相关文章

Vue3 的 shallowRef 和 shallowReactive:优化性能

大家对 Vue3 的 ref 和 reactive 都很熟悉,那么对 shallowRef 和 shallowReactive 是否了解呢? 在编程和数据结构中,“shallow”(浅层)通常指对数据结构的最外层进行操作,而不递归地处理其内部或嵌套的数据。这种处理方式关注的是数据结构的第一层属性或元素,而忽略更深层次的嵌套内容。 1. 浅层与深层的对比 1.1 浅层(Shallow) 定义

大模型研发全揭秘:客服工单数据标注的完整攻略

在人工智能(AI)领域,数据标注是模型训练过程中至关重要的一步。无论你是新手还是有经验的从业者,掌握数据标注的技术细节和常见问题的解决方案都能为你的AI项目增添不少价值。在电信运营商的客服系统中,工单数据是客户问题和解决方案的重要记录。通过对这些工单数据进行有效标注,不仅能够帮助提升客服自动化系统的智能化水平,还能优化客户服务流程,提高客户满意度。本文将详细介绍如何在电信运营商客服工单的背景下进行

基于MySQL Binlog的Elasticsearch数据同步实践

一、为什么要做 随着马蜂窝的逐渐发展,我们的业务数据越来越多,单纯使用 MySQL 已经不能满足我们的数据查询需求,例如对于商品、订单等数据的多维度检索。 使用 Elasticsearch 存储业务数据可以很好的解决我们业务中的搜索需求。而数据进行异构存储后,随之而来的就是数据同步的问题。 二、现有方法及问题 对于数据同步,我们目前的解决方案是建立数据中间表。把需要检索的业务数据,统一放到一张M

关于数据埋点,你需要了解这些基本知识

产品汪每天都在和数据打交道,你知道数据来自哪里吗? 移动app端内的用户行为数据大多来自埋点,了解一些埋点知识,能和数据分析师、技术侃大山,参与到前期的数据采集,更重要是让最终的埋点数据能为我所用,否则可怜巴巴等上几个月是常有的事。   埋点类型 根据埋点方式,可以区分为: 手动埋点半自动埋点全自动埋点 秉承“任何事物都有两面性”的道理:自动程度高的,能解决通用统计,便于统一化管理,但个性化定

性能测试介绍

性能测试是一种测试方法,旨在评估系统、应用程序或组件在现实场景中的性能表现和可靠性。它通常用于衡量系统在不同负载条件下的响应时间、吞吐量、资源利用率、稳定性和可扩展性等关键指标。 为什么要进行性能测试 通过性能测试,可以确定系统是否能够满足预期的性能要求,找出性能瓶颈和潜在的问题,并进行优化和调整。 发现性能瓶颈:性能测试可以帮助发现系统的性能瓶颈,即系统在高负载或高并发情况下可能出现的问题

使用SecondaryNameNode恢复NameNode的数据

1)需求: NameNode进程挂了并且存储的数据也丢失了,如何恢复NameNode 此种方式恢复的数据可能存在小部分数据的丢失。 2)故障模拟 (1)kill -9 NameNode进程 [lytfly@hadoop102 current]$ kill -9 19886 (2)删除NameNode存储的数据(/opt/module/hadoop-3.1.4/data/tmp/dfs/na

异构存储(冷热数据分离)

异构存储主要解决不同的数据,存储在不同类型的硬盘中,达到最佳性能的问题。 异构存储Shell操作 (1)查看当前有哪些存储策略可以用 [lytfly@hadoop102 hadoop-3.1.4]$ hdfs storagepolicies -listPolicies (2)为指定路径(数据存储目录)设置指定的存储策略 hdfs storagepolicies -setStoragePo

Hadoop集群数据均衡之磁盘间数据均衡

生产环境,由于硬盘空间不足,往往需要增加一块硬盘。刚加载的硬盘没有数据时,可以执行磁盘数据均衡命令。(Hadoop3.x新特性) plan后面带的节点的名字必须是已经存在的,并且是需要均衡的节点。 如果节点不存在,会报如下错误: 如果节点只有一个硬盘的话,不会创建均衡计划: (1)生成均衡计划 hdfs diskbalancer -plan hadoop102 (2)执行均衡计划 hd

10. 文件的读写

10.1 文本文件 操作文件三大类: ofstream:写操作ifstream:读操作fstream:读写操作 打开方式解释ios::in为了读文件而打开文件ios::out为了写文件而打开文件,如果当前文件存在则清空当前文件在写入ios::app追加方式写文件ios::trunc如果文件存在先删除,在创建ios::ate打开文件之后令读写位置移至文件尾端ios::binary二进制方式

性能分析之MySQL索引实战案例

文章目录 一、前言二、准备三、MySQL索引优化四、MySQL 索引知识回顾五、总结 一、前言 在上一讲性能工具之 JProfiler 简单登录案例分析实战中已经发现SQL没有建立索引问题,本文将一起从代码层去分析为什么没有建立索引? 开源ERP项目地址:https://gitee.com/jishenghua/JSH_ERP 二、准备 打开IDEA找到登录请求资源路径位置