U #FZhV@sxdZddlZddlZddlZddlZddlZddlZddlZ ddl m Z ddl m Z ddl mZddlmZdZdZd Zd Zeeejd Zd eZd ZdZdZdZejdej dZ!dZ"dZ#dZ$dZ%dZ&dZ'dZ(dZ)dZ*dZ+dZ,dZ-dZ.Gd d!d!e/Z0Gd"d#d#e0Z1Gd$d%d%e0Z2Gd&d'd'e0Z3Gd(d)d)e0Z4Gd*d+d+e0Z5d,d-Z6d.d/Z7d0d1Z8d2d3Z9d4d5Z:dS)6zVirtual bases classes for uploading media via Google APIs. Supported here are: * simple (media) uploads * multipart uploads that contain both metadata and a small file as payload * resumable uploads (with metadata as well) N)resumable_media)_helpers)common) ElementTreez content-typezbytes {:d}-{:d}/{:d}zbytes {:d}-{:d}/*z bytes */{:d}z==============={{:0{:d}d}}==s--s s3 content-type: application/json; charset=UTF-8 smultipart/related; boundary="zbytes=0-(?P\d+))flagszBytes stream is in unexpected state. The local stream has had {:d} bytes read from it while {:d} bytes have already been updated (they should match).zQ{:d} bytes have been read from the stream, which exceeds the expected total {:d}.DELETEPOSTPUTzjThe computed ``{}`` checksum, ``{}``, and the checksum reported by the remote host, ``{}``, did not match.zGResponse metadata had no ``{}`` value; checksum could not be validated.zFResponse headers had no ``{}`` value; checksum could not be validated.z?uploadsz'?partNumber={part}&uploadId={upload_id}z){http://s3.amazonaws.com/doc/2006-03-01/}ZUploadIdz?uploadId={upload_id}c@sReZdZdZdddZeddZddZed d Z ed d Z ed dZ dS) UploadBaseaBase class for upload helpers. Defines core shared behavior across different upload types. Args: upload_url (str): The URL where the content will be uploaded. headers (Optional[Mapping[str, str]]): Extra headers that should be sent with the request, e.g. headers for encrypted data. Attributes: upload_url (str): The URL where the content will be uploaded. NcCs,||_|dkri}||_d|_t|_dS)NF) upload_url_headers _finishedrZ RetryStrategyZ_retry_strategy)selfr headersre/home/aprabhat/apps/x.techxrdev.in/venv/lib/python3.8/site-packages/google/resumable_media/_upload.py__init__]s zUploadBase.__init__cCs|jS)z2bool: Flag indicating if the upload has completed.)rrrrrfinishedeszUploadBase.finishedcCs d|_t|tjjf|jdS)aProcess the response from an HTTP request. This is everything that must be done after a request that doesn't require network I/O (or other I/O). This is based on the `sans-I/O`_ philosophy. Args: response (object): The HTTP response object. Raises: ~google.resumable_media.common.InvalidResponse: If the status code is not 200. .. _sans-I/O: https://sans-io.readthedocs.io/ TN)rrrequire_status_codehttpclientOK_get_status_coderresponserrr_process_responsejszUploadBase._process_responsecCs tddS)zAccess the status code from an HTTP response. Args: response (object): The HTTP response object. Raises: NotImplementedError: Always, since virtual. This implementation is virtual.NNotImplementedErrorrrrrrs zUploadBase._get_status_codecCs tddS)zAccess the headers from an HTTP response. Args: response (object): The HTTP response object. Raises: NotImplementedError: Always, since virtual. rNrr!rrr _get_headerss zUploadBase._get_headerscCs tddS)zAccess the response body from an HTTP response. Args: response (object): The HTTP response object. Raises: NotImplementedError: Always, since virtual. rNrr!rrr _get_bodys zUploadBase._get_body)N) __name__ __module__ __qualname____doc__rpropertyrr staticmethodrr"r#rrrrr Os    r c@s"eZdZdZddZdddZdS) SimpleUploadaUpload a resource to a Google API. A **simple** media upload sends no metadata and completes the upload in a single request. Args: upload_url (str): The URL where the content will be uploaded. headers (Optional[Mapping[str, str]]): Extra headers that should be sent with the request, e.g. headers for encrypted data. Attributes: upload_url (str): The URL where the content will be uploaded. cCs@|jrtdt|ts&tdt|||jt<t|j ||jfS)aPrepare the contents of an HTTP request. This is everything that must be done before a request that doesn't require network I/O (or other I/O). This is based on the `sans-I/O`_ philosophy. .. note: This method will be used only once, so ``headers`` will be mutated by having a new key added to it. Args: data (bytes): The resource content to be uploaded. content_type (str): The content type for the request. Returns: Tuple[str, str, bytes, Mapping[str, str]]: The quadruple * HTTP verb for the request (always POST) * the URL for the request * the body of the request * headers for the request Raises: ValueError: If the current upload has already finished. TypeError: If ``data`` isn't bytes. .. _sans-I/O: https://sans-io.readthedocs.io/ An upload can only be used once.`data` must be bytes, received) r ValueError isinstancebytes TypeErrortyper _CONTENT_TYPE_HEADER_POSTr )rdata content_typerrr_prepare_requests   zSimpleUpload._prepare_requestNcCs tddS)a`Transmit the resource to be uploaded. Args: transport (object): An object which can make authenticated requests. data (bytes): The resource content to be uploaded. content_type (str): The content type of the resource, e.g. a JPEG image has content type ``image/jpeg``. timeout (Optional[Union[float, Tuple[float, float]]]): The number of seconds to wait for the server response. Depending on the retry strategy, a request may be repeated several times using the same timeout each time. Can also be passed as a tuple (connect_timeout, read_timeout). See :meth:`requests.Session.request` documentation for details. Raises: NotImplementedError: Always, since virtual. rNr)r transportr4r5timeoutrrrtransmitszSimpleUpload.transmit)N)r$r%r&r'r6r9rrrrr*s&r*cs4eZdZdZd fdd ZddZd ddZZS) MultipartUploada5Upload a resource with metadata to a Google API. A **multipart** upload sends both metadata and the resource in a single (multipart) request. Args: upload_url (str): The URL where the content will be uploaded. headers (Optional[Mapping[str, str]]): Extra headers that should be sent with the request, e.g. headers for encrypted data. checksum (Optional([str])): The type of checksum to compute to verify the integrity of the object. The request metadata will be amended to include the computed value. Using this option will override a manually-set checksum value. Supported values are "md5", "crc32c" and None. The default is None. Attributes: upload_url (str): The URL where the content will be uploaded. Ncstt|j||d||_dSNr)superr:r_checksum_type)rr rchecksum __class__rrrszMultipartUpload.__init__c Cs|jrtdt|ts&tdt|t|j}|dk rf| |t | }t |j}|||<t |||\}}t|d} | |jt<t|j||jfS)ajPrepare the contents of an HTTP request. This is everything that must be done before a request that doesn't require network I/O (or other I/O). This is based on the `sans-I/O`_ philosophy. .. note: This method will be used only once, so ``headers`` will be mutated by having a new key added to it. Args: data (bytes): The resource content to be uploaded. metadata (Mapping[str, str]): The resource metadata, such as an ACL list. content_type (str): The content type of the resource, e.g. a JPEG image has content type ``image/jpeg``. Returns: Tuple[str, str, bytes, Mapping[str, str]]: The quadruple * HTTP verb for the request (always POST) * the URL for the request * the body of the request * headers for the request Raises: ValueError: If the current upload has already finished. TypeError: If ``data`` isn't bytes. .. _sans-I/O: https://sans-io.readthedocs.io/ r+r,N")rr-r.r/r0r1r_get_checksum_objectr>updateprepare_checksum_digestdigest_get_metadata_keyconstruct_multipart_request_RELATED_HEADERr r2r3r ) rr4metadatar5Zchecksum_objectZactual_checksum metadata_keycontentmultipart_boundaryZmultipart_content_typerrrr6s$!      z MultipartUpload._prepare_requestcCs tddS)aTransmit the resource to be uploaded. Args: transport (object): An object which can make authenticated requests. data (bytes): The resource content to be uploaded. metadata (Mapping[str, str]): The resource metadata, such as an ACL list. content_type (str): The content type of the resource, e.g. a JPEG image has content type ``image/jpeg``. timeout (Optional[Union[float, Tuple[float, float]]]): The number of seconds to wait for the server response. Depending on the retry strategy, a request may be repeated several times using the same timeout each time. Can also be passed as a tuple (connect_timeout, read_timeout). See :meth:`requests.Session.request` documentation for details. Raises: NotImplementedError: Always, since virtual. rNr)rr7r4rJr5r8rrrr9>szMultipartUpload.transmit)NN)N)r$r%r&r'rr6r9 __classcell__rrr@rr:s6r:cseZdZdZd(fdd ZeddZeddZed d Zed d Z ed dZ d)ddZ ddZ d*ddZ ddZddZddZddZddZd+d d!Zd"d#Zd$d%Zd&d'ZZS),ResumableUploada#Initiate and fulfill a resumable upload to a Google API. A **resumable** upload sends an initial request with the resource metadata and then gets assigned an upload ID / upload URL to send bytes to. Using the upload URL, the upload is then done in chunks (determined by the user) until all bytes have been uploaded. Args: upload_url (str): The URL where the resumable upload will be initiated. chunk_size (int): The size of each chunk used to upload the resource. headers (Optional[Mapping[str, str]]): Extra headers that should be sent with every request. checksum (Optional([str])): The type of checksum to compute to verify the integrity of the object. After the upload is complete, the server-computed checksum of the resulting object will be read and google.resumable_media.common.DataCorruption will be raised on a mismatch. The corrupted file will not be deleted from the remote host automatically. Supported values are "md5", "crc32c" and None. The default is None. Attributes: upload_url (str): The URL where the content will be uploaded. Raises: ValueError: If ``chunk_size`` is not a multiple of :data:`.UPLOAD_CHUNK_SIZE`. Ncsvtt|j||d|tjdkr6tdtjd||_d|_d|_ d|_ d|_ ||_ d|_ d|_d|_d|_dS)Nr<rz{} KB must divide chunk sizeiF)r=rOrrZUPLOAD_CHUNK_SIZEr-format _chunk_size_stream _content_type_bytes_uploaded_bytes_checksummedr>_checksum_object _total_bytes_resumable_url_invalid)rr chunk_sizer?rr@rrrts"zResumableUpload.__init__cCs|jS)zbool: Indicates if the upload is in an invalid state. This will occur if a call to :meth:`transmit_next_chunk` fails. To recover from such a failure, call :meth:`recover`. rYrrrrinvalidszResumableUpload.invalidcCs|jS)z8int: The size of each chunk used to upload the resource.)rQrrrrrZszResumableUpload.chunk_sizecCs|jS)z;Optional[str]: The URL of the in-progress resumable upload.)rXrrrr resumable_urlszResumableUpload.resumable_urlcCs|jS)z-int: Number of bytes that have been uploaded.)rTrrrrbytes_uploadedszResumableUpload.bytes_uploadedcCs|jS)aOptional[int]: The total number of bytes to be uploaded. If this upload is initiated (via :meth:`initiate`) with ``stream_final=True``, this value will be populated based on the size of the ``stream`` being uploaded. (By default ``stream_final=True``.) If this upload is initiated with ``stream_final=False``, :attr:`total_bytes` will be :data:`None` since it cannot be determined from the stream. )rWrrrr total_bytess zResumableUpload.total_bytesTc Cs|jdk rtd|dkr&td||_||_tj|j}tj |j }d|ks^d|krn|j t |i}n|j t dd|i}|dk r||_ n|rt||_ |j dk rd |j } | |d <t|d } t|j| |fS) aPrepare the contents of HTTP request to initiate upload. This is everything that must be done before a request that doesn't require network I/O (or other I/O). This is based on the `sans-I/O`_ philosophy. Args: stream (IO[bytes]): The stream (i.e. file-like object) that will be uploaded. The stream **must** be at the beginning (i.e. ``stream.tell() == 0``). metadata (Mapping[str, str]): The resource metadata, such as an ACL list. content_type (str): The content type of the resource, e.g. a JPEG image has content type ``image/jpeg``. total_bytes (Optional[int]): The total number of bytes to be uploaded. If specified, the upload size **will not** be determined from the stream (even if ``stream_final=True``). stream_final (Optional[bool]): Indicates if the ``stream`` is "final" (i.e. no more bytes will be added to it). In this case we determine the upload size from the size of the stream. If ``total_bytes`` is passed, this argument will be ignored. Returns: Tuple[str, str, bytes, Mapping[str, str]]: The quadruple * HTTP verb for the request (always POST) * the URL for the request * the body of the request * headers for the request Raises: ValueError: If the current upload has already been initiated. ValueError: If ``stream`` is not at the beginning. .. _sans-I/O: https://sans-io.readthedocs.io/ N'This upload has already been initiated.rzStream must be at beginning.zx-goog-signaturezX-Goog-Signaturezapplication/json; charset=UTF-8zx-upload-content-typez{:d}zx-upload-content-lengthutf-8)r]r-tellrRrSurllibparseurlparser parse_qsqueryr r2rWget_total_bytesrPjsondumpsencoder3) rstreamrJr5r_ stream_finalZ parse_resultZ parsed_queryrcontent_lengthpayloadrrr_prepare_initiate_requests2'     z)ResumableUpload._prepare_initiate_requestcCs8tj|tjjtjjf|j|jdt|d|j |_ dS)aVProcess the response from an HTTP request that initiated upload. This is everything that must be done after a request that doesn't require network I/O (or other I/O). This is based on the `sans-I/O`_ philosophy. This method takes the URL from the ``Location`` header and stores it for future use. Within that URL, we assume the ``upload_id`` query parameter has been included, but we do not check. Args: response (object): The HTTP response object (need headers). .. _sans-I/O: https://sans-io.readthedocs.io/ callbacklocationN) rrrrrCREATEDr _make_invalidheader_requiredr"rXrrrr_process_initiate_responsesz*ResumableUpload._process_initiate_responsecCs tddS)aInitiate a resumable upload. By default, this method assumes your ``stream`` is in a "final" state ready to transmit. However, ``stream_final=False`` can be used to indicate that the size of the resource is not known. This can happen if bytes are being dynamically fed into ``stream``, e.g. if the stream is attached to application logs. If ``stream_final=False`` is used, :attr:`chunk_size` bytes will be read from the stream every time :meth:`transmit_next_chunk` is called. If one of those reads produces strictly fewer bites than the chunk size, the upload will be concluded. Args: transport (object): An object which can make authenticated requests. stream (IO[bytes]): The stream (i.e. file-like object) that will be uploaded. The stream **must** be at the beginning (i.e. ``stream.tell() == 0``). metadata (Mapping[str, str]): The resource metadata, such as an ACL list. content_type (str): The content type of the resource, e.g. a JPEG image has content type ``image/jpeg``. total_bytes (Optional[int]): The total number of bytes to be uploaded. If specified, the upload size **will not** be determined from the stream (even if ``stream_final=True``). stream_final (Optional[bool]): Indicates if the ``stream`` is "final" (i.e. no more bytes will be added to it). In this case we determine the upload size from the size of the stream. If ``total_bytes`` is passed, this argument will be ignored. timeout (Optional[Union[float, Tuple[float, float]]]): The number of seconds to wait for the server response. Depending on the retry strategy, a request may be repeated several times using the same timeout each time. Can also be passed as a tuple (connect_timeout, read_timeout). See :meth:`requests.Session.request` documentation for details. Raises: NotImplementedError: Always, since virtual. rNr)rr7rlrJr5r_rmr8rrrinitiates3zResumableUpload.initiatecCs|jrtd|jrtd|jdkr.tdt|j|j|j\}}}||jkrft ||j}t|| |||j t |jtj|i}t|j||fS)aPrepare the contents of HTTP request to upload a chunk. This is everything that must be done before a request that doesn't require network I/O. This is based on the `sans-I/O`_ philosophy. For the time being, this **does require** some form of I/O to read a chunk from ``stream`` (via :func:`get_next_chunk`). However, this will (almost) certainly not be network I/O. Returns: Tuple[str, str, bytes, Mapping[str, str]]: The quadruple * HTTP verb for the request (always PUT) * the URL for the request * the body of the request * headers for the request The headers incorporate the ``_headers`` on the current instance. Raises: ValueError: If the current upload has finished. ValueError: If the current upload is in an invalid state. ValueError: If the current upload has not been initiated. ValueError: If the location in the stream (i.e. ``stream.tell()``) does not agree with ``bytes_uploaded``. .. _sans-I/O: https://sans-io.readthedocs.io/ zUpload has finished.z;Upload is in an invalid state. To recover call `recover()`.Nz_This upload has not been initiated. Please call initiate() before beginning to transmit chunks.)rr-r\r]get_next_chunkrRrQrWr^_STREAM_ERROR_TEMPLATErP_update_checksumr r2rSrCONTENT_RANGE_HEADER_PUT)r start_bytero content_rangemsgrrrrr6Es4    z ResumableUpload._prepare_requestcCsf|js dS|jst|j|_||jkr@|j|}||d}n|}|j||jt|7_dS)a%Update the checksum with the payload if not already updated. Because error recovery can result in bytes being transmitted more than once, the checksum tracks the number of bytes checked in self._bytes_checksummed and skips bytes that have already been summed. N)r>rVrrCrUrDlen)rr~rooffsetr4rrrr{~s   z ResumableUpload._update_checksumcCs d|_dS)zSimple setter for ``invalid``. This is intended to be passed along as a callback to helpers that raise an exception so they can mark this instance as invalid before raising. TNr[rrrrruszResumableUpload._make_invalidcCstj|tjjtjjf|j|jd}|tjjkrL|j||_d|_ | |nVtj |tj |j |jd}t|}|dkr|t|d|dt|dd|_dS)aProcess the response from an HTTP request. This is everything that must be done after a request that doesn't require network I/O (or other I/O). This is based on the `sans-I/O`_ philosophy. Args: response (object): The HTTP response object. bytes_sent (int): The number of bytes sent in the request that ``response`` was returned for. Raises: ~google.resumable_media.common.InvalidResponse: If the status code is 308 and the ``range`` header is not of the form ``bytes 0-{end}``. ~google.resumable_media.common.InvalidResponse: If the status code is not 200 or 308. .. _sans-I/O: https://sans-io.readthedocs.io/ rqTNUnexpected "range" header*Expected to be of the form "bytes=0-{end}"end_byter)rrrrrPERMANENT_REDIRECTrrurTr_validate_checksumrv RANGE_HEADERr"_BYTES_RANGE_REmatchrInvalidResponseintgroup)rrZ bytes_sent status_code bytes_rangerrrr_process_resumable_responses4   z+ResumableUpload._process_resumable_responsecCs|jdkrdSt|j}|}||}|dkrNt|t|| |t |j }||krt |t|j||dS)aTCheck the computed checksum, if any, against the recieved metadata. Args: response (object): The HTTP response object. Raises: ~google.resumable_media.common.DataCorruption: If the checksum computed locally and the checksum reported by the remote host do not match. N)r>rrGrigetrr0_UPLOAD_METADATA_NO_APPROPRIATE_CHECKSUM_MESSAGErPr"rErVrFDataCorruption!_UPLOAD_CHECKSUM_MISMATCH_MESSAGEupper)rrrKrJremote_checksumlocal_checksumrrrrs.   z"ResumableUpload._validate_checksumcCs tddS)aTransmit the next chunk of the resource to be uploaded. If the current upload was initiated with ``stream_final=False``, this method will dynamically determine if the upload has completed. The upload will be considered complete if the stream produces fewer than :attr:`chunk_size` bytes when a chunk is read from it. Args: transport (object): An object which can make authenticated requests. timeout (Optional[Union[float, Tuple[float, float]]]): The number of seconds to wait for the server response. Depending on the retry strategy, a request may be repeated several times using the same timeout each time. Can also be passed as a tuple (connect_timeout, read_timeout). See :meth:`requests.Session.request` documentation for details. Raises: NotImplementedError: Always, since virtual. rNrrr7r8rrrtransmit_next_chunksz#ResumableUpload.transmit_next_chunkcCstjdi}t|jd|fS)a0Prepare the contents of HTTP request to recover from failure. This is everything that must be done before a request that doesn't require network I/O. This is based on the `sans-I/O`_ philosophy. We assume that the :attr:`resumable_url` is set (i.e. the only way the upload can end up :attr:`invalid` is if it has been initiated. Returns: Tuple[str, str, NoneType, Mapping[str, str]]: The quadruple * HTTP verb for the request (always PUT) * the URL for the request * the body of the request (always :data:`None`) * headers for the request The headers **do not** incorporate the ``_headers`` on the current instance. .. _sans-I/O: https://sans-io.readthedocs.io/ z bytes */*N)rr|r}r])rrrrr_prepare_recover_requests z(ResumableUpload._prepare_recover_requestcCst|tjjf|j||}tj|krl|tj}t |}|dkrVt |d|dt | dd|_nd|_|j|jd|_dS)aProcess the response from an HTTP request to recover from failure. This is everything that must be done after a request that doesn't require network I/O (or other I/O). This is based on the `sans-I/O`_ philosophy. Args: response (object): The HTTP response object. Raises: ~google.resumable_media.common.InvalidResponse: If the status code is not 308. ~google.resumable_media.common.InvalidResponse: If the status code is 308 and the ``range`` header is not of the form ``bytes 0-{end}``. .. _sans-I/O: https://sans-io.readthedocs.io/ NrrrrrF)rrrrrrr"rrrrrrrrTrRseekrY)rrrrrrrr_process_recover_response*s(    z)ResumableUpload._process_recover_responsecCs tddS)a!Recover from a failure. This method should be used when a :class:`ResumableUpload` is in an :attr:`~ResumableUpload.invalid` state due to a request failure. This will verify the progress with the server and make sure the current upload is in a valid state before :meth:`transmit_next_chunk` can be used again. Args: transport (object): An object which can make authenticated requests. Raises: NotImplementedError: Always, since virtual. rNr)rr7rrrrecoverSszResumableUpload.recover)NN)NT)NTN)N)r$r%r&r'rr(r\rZr]r^r_rprwrxr6r{rurrrrrrrNrrr@rrOWs:      I  59 ;! )rOcseZdZdZdfdd ZeddZddZd d Zd d Z dd dZ ddZ ddZ dddZ ddZddZdddZZS)XMLMPUContaineralInitiate and close an upload using the XML MPU API. An XML MPU sends an initial request and then receives an upload ID. Using the upload ID, the upload is then done in numbered parts and the parts can be uploaded concurrently. In order to avoid concurrency issues with this container object, the uploading of individual parts is handled separately, by XMLMPUPart objects spawned from this container class. The XMLMPUPart objects are not necessarily in the same process as the container, so they do not update the container automatically. MPUs are sometimes referred to as "Multipart Uploads", which is ambiguous given the JSON multipart upload, so the abbreviation "MPU" will be used throughout. See: https://cloud.google.com/storage/docs/multipart-uploads Args: upload_url (str): The URL of the object (without query parameters). The initiate, PUT, and finalization requests will all use this URL, with varying query parameters. filename (str): The name (path) of the file to upload. headers (Optional[Mapping[str, str]]): Extra headers that should be sent with every request. Attributes: upload_url (str): The URL where the content will be uploaded. upload_id (Optional(str)): The ID of the upload from the initialization response. Ncs&tj||d||_||_i|_dSr;)r=r _filename _upload_id_parts)rr filenamer upload_idr@rrrszXMLMPUContainer.__init__cCs|jSNrrrrrrszXMLMPUContainer.upload_idcCs||j|<dS)aRegister an uploaded part by part number and corresponding etag. XMLMPUPart objects represent individual parts, and their part number and etag can be registered to the container object with this method and therefore incorporated in the finalize() call to finish the upload. This method accepts part_number and etag, but not XMLMPUPart objects themselves, to reduce the complexity involved in running XMLMPUPart uploads in separate processes. Args: part_number (int): The part number. Parts are assembled into the final uploaded object with finalize() in order of their part numbers. etag (str): The etag included in the server response after upload. N)r)r part_numberetagrrr register_partszXMLMPUContainer.register_partcCs6|jdk rtd|jt}|jt|i}t|d|fS)aPrepare the contents of HTTP request to initiate upload. This is everything that must be done before a request that doesn't require network I/O (or other I/O). This is based on the `sans-I/O`_ philosophy. Args: content_type (str): The content type of the resource, e.g. a JPEG image has content type ``image/jpeg``. Returns: Tuple[str, str, bytes, Mapping[str, str]]: The quadruple * HTTP verb for the request (always POST) * the URL for the request * the body of the request * headers for the request Raises: ValueError: If the current upload has already been initiated. .. _sans-I/O: https://sans-io.readthedocs.io/ Nr`)rr-r _MPU_INITIATE_QUERYr r2r3)rr5Z initiate_urlrrrrrps  z)XMLMPUContainer._prepare_initiate_requestcCs8t|tjjf|jt|j}| t t j|_ dS)aProcess the response from an HTTP request that initiated the upload. This is everything that must be done after a request that doesn't require network I/O (or other I/O). This is based on the `sans-I/O`_ philosophy. This method takes the URL from the ``Location`` header and stores it for future use. Within that URL, we assume the ``upload_id`` query parameter has been included, but we do not check. Args: response (object): The HTTP response object. Raises: ~google.resumable_media.common.InvalidResponse: If the status code is not 200. .. _sans-I/O: https://sans-io.readthedocs.io/ N) rrrrrrr fromstringtextfind_S3_COMPAT_XML_NAMESPACE_UPLOAD_ID_NODEr)rrrootrrrrws z*XMLMPUContainer._process_initiate_responsecCs tddS)a%Initiate an MPU and record the upload ID. Args: transport (object): An object which can make authenticated requests. content_type (str): The content type of the resource, e.g. a JPEG image has content type ``image/jpeg``. timeout (Optional[Union[float, Tuple[float, float]]]): The number of seconds to wait for the server response. Depending on the retry strategy, a request may be repeated several times using the same timeout each time. Can also be passed as a tuple (connect_timeout, read_timeout). See :meth:`requests.Session.request` documentation for details. Raises: NotImplementedError: Always, since virtual. rNr)rr7r5r8rrrrxszXMLMPUContainer.initiatecCs|jdkrtdtj|jd}|j|}td}|j D]4\}}t |d}t |t |d_ |t |d_ q>t |}t|||jfS)aPrepare the contents of an HTTP request to finalize the upload. All of the parts must be registered before calling this method. Returns: Tuple[str, str, bytes, Mapping[str, str]]: The quadruple * HTTP verb for the request (always POST) * the URL for the request * the body of the request * headers for the request Raises: ValueError: If the upload has not been initiated. N'This upload has not yet been initiated.rZCompleteMultipartUploadZPartZ PartNumberETag)rr-_MPU_FINAL_QUERY_TEMPLATErPrr rZElementritemsZ SubElementstrrtostringr3r )rZ final_queryZ finalize_urlZfinal_xml_rootrrpartrorrr_prepare_finalize_requests     z)XMLMPUContainer._prepare_finalize_requestcCs t|tjjf|jd|_dS)aProcess the response from an HTTP request that finalized the upload. This is everything that must be done after a request that doesn't require network I/O (or other I/O). This is based on the `sans-I/O`_ philosophy. Args: response (object): The HTTP response object. Raises: ~google.resumable_media.common.InvalidResponse: If the status code is not 200. .. _sans-I/O: https://sans-io.readthedocs.io/ TN)rrrrrrrrrrr_process_finalize_responsesz*XMLMPUContainer._process_finalize_responsecCs tddS)aFinalize an MPU request with all the parts. Args: transport (object): An object which can make authenticated requests. timeout (Optional[Union[float, Tuple[float, float]]]): The number of seconds to wait for the server response. Depending on the retry strategy, a request may be repeated several times using the same timeout each time. Can also be passed as a tuple (connect_timeout, read_timeout). See :meth:`requests.Session.request` documentation for details. Raises: NotImplementedError: Always, since virtual. rNrrrrrfinalize+szXMLMPUContainer.finalizecCs8|jdkrtdtj|jd}|j|}t|d|jfS)aPrepare the contents of an HTTP request to cancel the upload. Returns: Tuple[str, str, bytes, Mapping[str, str]]: The quadruple * HTTP verb for the request (always DELETE) * the URL for the request * the body of the request * headers for the request Raises: ValueError: If the upload has not been initiated. Nrr)rr-rrPrr _DELETEr )rZ cancel_queryZ cancel_urlrrr_prepare_cancel_requestBs   z'XMLMPUContainer._prepare_cancel_requestcCst|tjjf|jdS)aProcess the response from an HTTP request that canceled the upload. This is everything that must be done after a request that doesn't require network I/O (or other I/O). This is based on the `sans-I/O`_ philosophy. Args: response (object): The HTTP response object. Raises: ~google.resumable_media.common.InvalidResponse: If the status code is not 204. .. _sans-I/O: https://sans-io.readthedocs.io/ N)rrrr NO_CONTENTrrrrr_process_cancel_responseWs z(XMLMPUContainer._process_cancel_responsecCs tddS)aCancel an MPU request and permanently delete any uploaded parts. This cannot be undone. Args: transport (object): An object which can make authenticated requests. timeout (Optional[Union[float, Tuple[float, float]]]): The number of seconds to wait for the server response. Depending on the retry strategy, a request may be repeated several times using the same timeout each time. Can also be passed as a tuple (connect_timeout, read_timeout). See :meth:`requests.Session.request` documentation for details. Raises: NotImplementedError: Always, since virtual. rNrrrrrcancellszXMLMPUContainer.cancel)NN)N)N)N)r$r%r&r'rr(rrrprwrxrrrrrrrNrrr@rrgs"  #  rcseZdZdZdfdd ZeddZeddZed d Zed d Z ed dZ eddZ ddZ ddZ dddZddZZS) XMLMPUPartaUpload a single part of an existing XML MPU container. An XML MPU sends an initial request and then receives an upload ID. Using the upload ID, the upload is then done in numbered parts and the parts can be uploaded concurrently. In order to avoid concurrency issues with the container object, the uploading of individual parts is handled separately by multiple objects of this class. Once a part is uploaded, it can be registered with the container with `container.register_part(part.part_number, part.etag)`. MPUs are sometimes referred to as "Multipart Uploads", which is ambiguous given the JSON multipart upload, so the abbreviation "MPU" will be used throughout. See: https://cloud.google.com/storage/docs/multipart-uploads Args: upload_url (str): The URL of the object (without query parameters). upload_id (str): The ID of the upload from the initialization response. filename (str): The name (path) of the file to upload. start (int): The byte index of the beginning of the part. end (int): The byte index of the end of the part. part_number (int): The part number. Part numbers will be assembled in sequential order when the container is finalized. headers (Optional[Mapping[str, str]]): Extra headers that should be sent with every request. checksum (Optional([str])): The type of checksum to compute to verify the integrity of the object. The request headers will be amended to include the computed value. Supported values are "md5", "crc32c" and None. The default is None. Attributes: upload_url (str): The URL of the object (without query parameters). upload_id (str): The ID of the upload from the initialization response. filename (str): The name (path) of the file to upload. start (int): The byte index of the beginning of the part. end (int): The byte index of the end of the part. part_number (int): The part number. Part numbers will be assembled in sequential order when the container is finalized. etag (Optional(str)): The etag returned by the service after upload. Nc sDtj||d||_||_||_||_||_d|_||_d|_ dSr;) r=rr_start_endr _part_number_etagr>rV) rr rrstartendrrr?r@rrrs zXMLMPUPart.__init__cCs|jSr)rrrrrrszXMLMPUPart.part_numbercCs|jSrrrrrrrszXMLMPUPart.upload_idcCs|jSr)rrrrrrszXMLMPUPart.filenamecCs|jSr)rrrrrrszXMLMPUPart.etagcCs|jSr)rrrrrrszXMLMPUPart.startcCs|jSr)rrrrrrszXMLMPUPart.endc Cs|jrtdt|jd$}||j||j|j}W5QRXt |j |_ |j dk rh|j |t j|j|jd}|j|}t|||jfS)a|Prepare the contents of HTTP request to upload a part. This is everything that must be done before a request that doesn't require network I/O. This is based on the `sans-I/O`_ philosophy. For the time being, this **does require** some form of I/O to read a part from ``stream`` (via :func:`get_part_payload`). However, this will (almost) certainly not be network I/O. Returns: Tuple[str, str, bytes, Mapping[str, str]]: The quadruple * HTTP verb for the request (always PUT) * the URL for the request * the body of the request * headers for the request The headers incorporate the ``_headers`` on the current instance. Raises: ValueError: If the current upload has finished. .. _sans-I/O: https://sans-io.readthedocs.io/ z$This part has already been uploaded.brN)rr)rr-openrrrreadrrrCr>rVrD_MPU_PART_QUERY_TEMPLATErPrrr r}r )rfroZ part_queryr rrr_prepare_upload_requests    z"XMLMPUPart._prepare_upload_requestcCs@t|tjjf|j||t|d|j}||_ d|_ dS)aProcess the response from an HTTP request. This is everything that must be done after a request that doesn't require network I/O (or other I/O). This is based on the `sans-I/O`_ philosophy. Args: response (object): The HTTP response object. Raises: ~google.resumable_media.common.InvalidResponse: If the status code is not 200 or the response is missing data. .. _sans-I/O: https://sans-io.readthedocs.io/ rTN) rrrrrrrrvr"rr)rrrrrr_process_upload_response s z#XMLMPUPart._process_upload_responsecCs tddS)aUpload the part. Args: transport (object): An object which can make authenticated requests. timeout (Optional[Union[float, Tuple[float, float]]]): The number of seconds to wait for the server response. Depending on the retry strategy, a request may be repeated several times using the same timeout each time. Can also be passed as a tuple (connect_timeout, read_timeout). See :meth:`requests.Session.request` documentation for details. Raises: NotImplementedError: Always, since virtual. rNrrrrrupload%szXMLMPUPart.uploadcCs|jdkrdSt||j|j}|dkrNt|j}t|t|||t |j }||krt |t |j||dS)aSCheck the computed checksum, if any, against the response headers. Args: response (object): The HTTP response object. Raises: ~google.resumable_media.common.DataCorruption: If the checksum computed locally and the checksum reported by the remote host do not match. N)r>rZ#_get_uploaded_checksum_from_headersr"rGrrrrPrErVrFrrr)rrrrKrrrrr<s4  zXMLMPUPart._validate_checksum)NN)N)r$r%r&r'rr(rrrrrrrrrrrNrrr@rrs*3      * rcCs ttj}t|}|dS)zGet a random boundary for a multipart request. Returns: bytes: The boundary used to separate parts of a multipart request. ra)random randrangesysmaxsize_BOUNDARY_FORMATrPrk)Z random_intboundaryrrr get_boundaryas  rcCsht}t|d}|d}t|}|t|t|td|tt|t|t}||fS)aConstruct a multipart request body. Args: data (bytes): The resource content (UTF-8 encoded as bytes) to be uploaded. metadata (Mapping[str, str]): The resource metadata, such as an ACL list. content_type (str): The content type of the resource, e.g. a JPEG image has content type ``image/jpeg``. Returns: Tuple[bytes, bytes]: The multipart request body and the boundary used between each part. rascontent-type: )rrirjrk_MULTIPART_SEP_MULTIPART_BEGIN_CRLF)r4rJr5rMZ json_bytesZ boundary_seprLrrrrHnsB      rHcCs,|}|dtj|}|||S)zDetermine the total number of bytes in a stream. Args: stream (IO[bytes]): The stream (i.e. file-like object). Returns: int: The number of bytes. r)rbrosSEEK_END)rlcurrent_positionZ end_positionrrrrhs  rhcCs|}|dk r<|||kr(dkrst      ULg!\ )9