EXPDP/IMPDP 中的并行度PARALLEL参数

2023-10-08 00:48

本文主要是介绍EXPDP/IMPDP 中的并行度PARALLEL参数,希望对大家解决编程问题提供一定的参考价值,需要的开发者们随着小编来一起学习吧!

    如果设置 EXPDP parallel=4 必须要设置4个EXPDP文件,不然PARALLEL是有问题的,同时EXPDP会使用一个WORKER进程导出METADATA,其他WORKER进程会同时出数据。如果EXPDP作业低于250M,只会启动一个WORKER进程。如果是500M会启动2个,1000M及会启动4个WOKER进程。一般来说加上%U来设置多个文件。
    而IMPDP有所不同,会先启动一个WOKER进程METADATA导入,然后启动多个WORKER进程导入,所以再前期只会看到WOKER在导入METADATA,而且IMPDP如果PARALLE=4也需要不少于4个DMP文件,也可以使用%U来进行导入。
nohup expdp system/**** PARALLEL=2 JOB_NAME=full_bak_job full=y dumpfile=exptest:back_%U.dmp logfile=exptest:back.log &
impdp system/*** PARALLEL=2 EXCLUDE=STATISTICS JOB_NAME=full_imp cluster=no full=y dumpfile=test:back_%U.dmp logfile=test:back_imp.log;
    而在11GR2后EXPDP和IMDP的WORKER进程会在多个INSTANCE启动,所以DIRECTORY必须在共享磁盘上,如果没有设置共享磁盘还是指定cluster=no 来防止报错。
当观察EXPDP/IMPDP woker的时候如下:
Import> status
Job: FULL_IMP
  Operation: IMPORT                        
  Mode: FULL                          
  State: EXECUTING                     
  Bytes Processed: 150,300,713,536
  Percent Done: 80
  Current Parallelism: 6
  Job Error Count: 0
  Dump File: /expdp/back_%u.dmp
  Dump File: /expdp/back_01.dmp
  Dump File: /expdp/back_02.dmp
  Dump File: /expdp/back_03.dmp
  Dump File: /expdp/back_04.dmp
  Dump File: /expdp/back_05.dmp
  Dump File: /expdp/back_06.dmp
  Dump File: /expdp/back_07.dmp
  Dump File: /expdp/back_08.dmp
 
Worker 1 Status:
  Process Name: DW00
  State: EXECUTING                     
  Object Schema: ACRUN
  Object Name: T_PLY_UNDRMSG
  Object Type: DATABASE_EXPORT/SCHEMA/TABLE/TABLE_DATA
  Completed Objects: 3
  Completed Rows: 3,856,891
  Completed Bytes: 1,134,168,200
  Percent Done: 83
  Worker Parallelism: 1
 
Worker 2 Status:
  Process Name: DW01
  State: EXECUTING                     
  Object Schema: ACRUN
  Object Name: T_FIN_PAYDUE
  Object Type: DATABASE_EXPORT/SCHEMA/TABLE/TABLE_DATA
  Completed Objects: 5
  Completed Rows: 2,646,941
  Completed Bytes: 1,012,233,224
  Percent Done: 93
  Worker Parallelism: 1
 
Worker 3 Status:
  Process Name: DW02
  State: EXECUTING                     
  Object Schema: ACRUN
  Object Name: MLOG$_T_FIN_CLMDUE
  Object Type: DATABASE_EXPORT/SCHEMA/TABLE/TABLE_DATA
  Completed Objects: 6
  Completed Bytes: 382,792,584
  Worker Parallelism: 1
 
Worker 4 Status:
  Process Name: DW03
  State: EXECUTING                     
  Object Schema: ACRUN
  Object Name: T_PAY_CONFIRM_INFO
  Object Type: DATABASE_EXPORT/SCHEMA/TABLE/TABLE_DATA
  Completed Objects: 5
  Completed Rows: 2,443,790
  Completed Bytes: 943,310,104
  Percent Done: 83
  Worker Parallelism: 1
 
Worker 5 Status:
  Process Name: DW04
  State: EXECUTING                     
  Object Schema: ACRUN
  Object Name: T_PLY_TGT
  Object Type: DATABASE_EXPORT/SCHEMA/TABLE/TABLE_DATA
  Completed Objects: 6
  Completed Rows: 2,285,353
  Completed Bytes: 822,501,496
  Percent Done: 64
  Worker Parallelism: 1
 
Worker 6 Status:
  Process Name: DW05
  State: EXECUTING                     
  Object Schema: ACRUN
  Object Name: T_FIN_PREINDRCT_CLMFEE
  Object Type: DATABASE_EXPORT/SCHEMA/TABLE/TABLE_DATA
  Completed Objects: 5
  Completed Rows: 6,042,384
  Completed Bytes: 989,435,088
  Percent Done: 79
  Worker Parallelism: 1
 
英文如下:
    For Data Pump Export, the value that is specified for the parallel parameter should be less than or equal to the number of files in the dump file set. Each worker or Parallel Execution Process requires exclusive access to the dump file, so having fewer dump files than the degree of parallelism will mean that some workers or PX processes will be unable to write the information they are exporting. If this occurs, the worker processes go into an idle state and will not be doing any work until more files are added to the job. See the explanation of the DUMPFILE parameter in the Database Utilities guide for details on how to specify multiple dump files for a Data Pump export job.
    For Data Pump Import, the workers and PX processes can all read from the same files. However, if there are not enough dump files, the performance may not be optimal because multiple threads of execution will be trying to access the same dump file. The performance impact of multiple processes sharing the dump files depends on the I/O subsystem containing the dump files. For this reason, Data Pump Import should not have a value for the PARALLEL parameter that is significantly larger than the number of files in the dump file set.
 
     In a typical export that includes both data and metadata, the first worker process will unload the metadata: tablespaces, schemas, grants, roles, tables, indexes, and so on. This single worker unloads the metadata, and all the rest unload the data, all at the same time. If the metadata worker finishes and there are still data objects to unload, it will start unloading the data too. The examples in this document assume that there is always one worker busy unloading metadata while the rest of the workers are busy unloading table data objects.
 
    If the external tables method is chosen, Data Pump will determine the maximum number of PX processes that can work on a table data object. It does this by dividing the estimated size of the table data object by 250 MB and rounding the result down. If the result is zero or one, then PX processes are not used to unload the table
 
    The PARALLEL parameter works a bit differently in Import than Export. Because there are various dependencies that exist when creating objects during import, everything must be done in order. For Import, no data loading can occur until the tables are created because data cannot be loaded into tables that do not yet exist

这篇关于EXPDP/IMPDP 中的并行度PARALLEL参数的文章就介绍到这儿,希望我们推荐的文章对编程师们有所帮助!



http://www.chinasem.cn/article/161563

相关文章

Java通过反射获取方法参数名的方式小结

《Java通过反射获取方法参数名的方式小结》这篇文章主要为大家详细介绍了Java如何通过反射获取方法参数名的方式,文中的示例代码讲解详细,感兴趣的小伙伴可以跟随小编一起学习一下... 目录1、前言2、解决方式方式2.1: 添加编译参数配置 -parameters方式2.2: 使用Spring的内部工具类 -

Python调用另一个py文件并传递参数常见的方法及其应用场景

《Python调用另一个py文件并传递参数常见的方法及其应用场景》:本文主要介绍在Python中调用另一个py文件并传递参数的几种常见方法,包括使用import语句、exec函数、subproce... 目录前言1. 使用import语句1.1 基本用法1.2 导入特定函数1.3 处理文件路径2. 使用ex

MySQL中时区参数time_zone解读

《MySQL中时区参数time_zone解读》MySQL时区参数time_zone用于控制系统函数和字段的DEFAULTCURRENT_TIMESTAMP属性,修改时区可能会影响timestamp类型... 目录前言1.时区参数影响2.如何设置3.字段类型选择总结前言mysql 时区参数 time_zon

Python如何使用seleniumwire接管Chrome查看控制台中参数

《Python如何使用seleniumwire接管Chrome查看控制台中参数》文章介绍了如何使用Python的seleniumwire库来接管Chrome浏览器,并通过控制台查看接口参数,本文给大家... 1、cmd打开控制台,启动谷歌并制定端口号,找不到文件的加环境变量chrome.exe --rem

Linux中Curl参数详解实践应用

《Linux中Curl参数详解实践应用》在现代网络开发和运维工作中,curl命令是一个不可或缺的工具,它是一个利用URL语法在命令行下工作的文件传输工具,支持多种协议,如HTTP、HTTPS、FTP等... 目录引言一、基础请求参数1. -X 或 --request2. -d 或 --data3. -H 或

Oracle Expdp按条件导出指定表数据的方法实例

《OracleExpdp按条件导出指定表数据的方法实例》:本文主要介绍Oracle的expdp数据泵方式导出特定机构和时间范围的数据,并通过parfile文件进行条件限制和配置,文中通过代码介绍... 目录1.场景描述 2.方案分析3.实验验证 3.1 parfile文件3.2 expdp命令导出4.总结

详解Spring Boot接收参数的19种方式

《详解SpringBoot接收参数的19种方式》SpringBoot提供了多种注解来接收不同类型的参数,本文给大家介绍SpringBoot接收参数的19种方式,感兴趣的朋友跟随小编一起看看吧... 目录SpringBoot接受参数相关@PathVariable注解@RequestHeader注解@Reque

Java向kettle8.0传递参数的方式总结

《Java向kettle8.0传递参数的方式总结》介绍了如何在Kettle中传递参数到转换和作业中,包括设置全局properties、使用TransMeta和JobMeta的parameterValu... 目录1.传递参数到转换中2.传递参数到作业中总结1.传递参数到转换中1.1. 通过设置Trans的

java如何调用kettle设置变量和参数

《java如何调用kettle设置变量和参数》文章简要介绍了如何在Java中调用Kettle,并重点讨论了变量和参数的区别,以及在Java代码中如何正确设置和使用这些变量,避免覆盖Kettle中已设置... 目录Java调用kettle设置变量和参数java代码中变量会覆盖kettle里面设置的变量总结ja

spring 参数校验Validation示例详解

《spring参数校验Validation示例详解》Spring提供了Validation工具类来实现对客户端传来的请求参数的有效校验,本文给大家介绍spring参数校验Validation示例详... 目录前言一、Validation常见的校验注解二、Validation的简单应用三、分组校验四、自定义校