U EZh8@stddlZddlZddlmZmZmZmZddlmZm Z m Z m Z GdddeZ GdddeZ Gdd d eZdS) N)CompleteMultipartUploadTaskCreateMultipartUploadTaskSubmissionTaskTask)ChunksizeAdjustercalculate_range_parameter get_callbacksget_filtered_dictc @seZdZdZdddddddd d d Zd d ddddddddd d g Zd d dddddddg Zdddd d gZddZddZ ddZ ddZ ddZ dd Z d!d"Zd#S)$CopySubmissionTaskz+Task for submitting tasks to execute a copyZIfMatchZIfModifiedSinceZ IfNoneMatchZIfUnmodifiedSinceZSSECustomerKeyZSSECustomerAlgorithmZSSECustomerKeyMD5 RequestPayerExpectedBucketOwner) CopySourceIfMatchCopySourceIfModifiedSinceCopySourceIfNoneMatchCopySourceIfUnmodifiedSinceCopySourceSSECustomerKeyCopySourceSSECustomerAlgorithmCopySourceSSECustomerKeyMD5r r r rrrrrrZMetadataDirectiveZTaggingDirectivec Cs|jjdkrn|jj}||j}|j}|D] \} } | |jkr.| ||j| <q.|jj f|} |j | d|jj|j kr| |||||n| |||||dS)a :param client: The client associated with the transfer manager :type config: s3transfer.manager.TransferConfig :param config: The transfer config associated with the transfer manager :type osutil: s3transfer.utils.OSUtil :param osutil: The os utility associated to the transfer manager :type request_executor: s3transfer.futures.BoundedExecutor :param request_executor: The request executor associated with the transfer manager :type transfer_future: s3transfer.futures.TransferFuture :param transfer_future: The transfer future associated with the transfer request that tasks are being submitted for NZ ContentLength)metasize call_args)_get_head_object_request_from_copy_source copy_source extra_argsitemsEXTRA_ARGS_TO_HEAD_ARGS_MAPPINGZ source_clientZ head_objectZprovide_transfer_sizeZmultipart_threshold_submit_copy_request_submit_multipart_request) selfclientconfigosutilrequest_executortransfer_futurerZhead_object_requestrparamvalueresponser'X/home/aprabhat/apps/x.techxrdev.in/venv/lib/python3.8/site-packages/s3transfer/copies.py_submitPsD  zCopySubmissionTask._submitc CsL|jj}t|d}|j|t|j||j|j|j|j ||jj ddddS)Nprogress)rrbucketkeyr callbacksrT)transfer_coordinator main_kwargsis_final) rrr_transfer_coordinatorsubmitCopyObjectTaskrr+r,rr)rrr r!r"r#rprogress_callbacksr'r'r(rs"  z'CopySubmissionTask._submit_copy_requestcCsx|jj}i}|jD]\}} ||jkr| ||<q|j|t|j||j|j |dd} |j } t } | | |jj } tt|jj t| } g}t|d}td| dD]}||j}t| |d| |jj |d<|| |d| |jj }|jd}||j|t|j||j|j|j |||||d d| id q||j}|j|t|j||j|j |d| |d d d dS) N)rr+r,r)r.r/r*ZCopySourceRangeZChecksumAlgorithm) rrr+r, part_numberrr-rchecksum_algorithm upload_id)r.r/pending_main_kwargs)r8partsT)r.r/r9r0)rrrrCREATE_MULTIPART_ARGS_BLACKLISTr1r2rr+r,Zmultipart_chunksizerZadjust_chunksizerintmathceilfloatrrange_extra_upload_part_argsr_get_transfer_sizegetappend CopyPartTaskr_extra_complete_multipart_argsr)rrr r!r"r#rZcreate_multipart_extra_argsr$valZcreate_multipart_future part_sizeZadjuster num_partsZ part_futuresr4r6Zextra_part_argsrr7Zcomplete_multipart_extra_argsr'r'r(rs     z,CopySubmissionTask._submit_multipart_requestcCs2t|trt|Std|dt|ddS)NzLExpecting dictionary formatted: {"Bucket": bucket_name, "Key": key} but got z or type .) isinstancedictcopy TypeErrortype)rrr'r'r(rs   zl r c@seZdZdZddZdS)r3zTask to do a nonmultipart copyc Cs0|jf|||d||D]}||dqdS)a :param client: The client to use when calling PutObject :param copy_source: The CopySource parameter to use :param bucket: The name of the bucket to copy to :param key: The name of the key to copy to :param extra_args: A dictionary of any extra arguments that may be used in the upload. :param callbacks: List of callbacks to call after copy :param size: The size of the transfer. This value is passed into the callbacks ) CopySourceBucketKeyZbytes_transferredN)Z copy_object) rrrr+r,rr-rcallbackr'r'r(_main3szCopyObjectTask._mainNrTrUrVrWr]r'r'r'r(r30sr3c@seZdZdZdddZdS)rEz)Task to upload a part in a multipart copyNc Csx|jf|||||d|} |D]} | | dq | dd} | |d}| rtd| }|| dkrt| d|||<|S)a< :param client: The client to use when calling PutObject :param copy_source: The CopySource parameter to use :param bucket: The name of the bucket to upload to :param key: The name of the key to upload to :param upload_id: The id of the upload :param part_number: The number representing the part of the multipart upload :param extra_args: A dictionary of any extra arguments that may be used in the upload. :param callbacks: List of callbacks to call after copy part :param size: The size of the transfer. This value is passed into the callbacks :param checksum_algorithm: The algorithm that was used to create the multipart upload :rtype: dict :returns: A dictionary representing a part:: {'Etag': etag_value, 'PartNumber': part_number} This value can be appended to a list to be used to complete the multipart upload. If a checksum is in the response, it will also be included. )rXrYrZZUploadId PartNumberr[ZCopyPartResultETag)r`r_ZChecksum)Zupload_part_copyupper)rrrr+r,r8r6rr-rr7r&r\etagZ part_metadataZchecksum_memberr'r'r(r]Ls(&    zCopyPartTask._main)Nr^r'r'r'r(rEIs rE)rMr=Zs3transfer.tasksrrrrZs3transfer.utilsrrrr r r3rEr'r'r'r( s