U $FZhkZ@s`dZddlZddlZddlmZddlZddlmZddlmZddl m Z ddl m Z ddl mZdd lmZdd lmZd Zd Zd ZdZdZejedZdZeedZdZdZejjZej j!Z"ddZ#ddZ$ddZ%d:ddZ&ddZ'd d!Z(d"d#Z)Gd$d%d%e*Z+d&d'Z,d;d)d*Z-d+d,Z.d-d.Z/d/d0Z0d1d2Z1dzVHelper functions for Cloud Storage utility classes. These are *not* part of the API. N)md5)urlsplit) urlunsplit)uuid4)environment_vars)_DEFAULT_TIMEOUT) DEFAULT_RETRY))DEFAULT_RETRY_IF_METAGENERATION_SPECIFIEDZSTORAGE_EMULATOR_HOSTZAPI_ENDPOINT_OVERRIDEZAPI_VERSION_OVERRIDEzgoogleapis.comzstorage.{universe_domain})Zuniverse_domainzhttps://Zv1)) if_etag_matchzIf-Match)if_etag_not_matchz If-None-Match))if_generation_matchZifGenerationMatch)if_generation_not_matchZifGenerationNotMatch)if_metageneration_matchZifMetagenerationMatch)if_metageneration_not_matchZifMetagenerationNotMatch)Zif_source_generation_matchZifSourceGenerationMatch)Zif_source_generation_not_matchZifSourceGenerationNotMatch)Zif_source_metageneration_matchZifSourceMetagenerationMatch)Z"if_source_metageneration_not_matchZifSourceMetagenerationNotMatchcCstjtdSN)osenvirongetSTORAGE_EMULATOR_ENV_VARrrd/home/aprabhat/apps/x.techxrdev.in/venv/lib/python3.8/site-packages/google/cloud/storage/_helpers.py_get_storage_emulator_overrideQsrcCsttttSr)rgetenv_API_ENDPOINT_OVERRIDE_ENV_VAR_DEFAULT_SCHEME_TRUE_DEFAULT_STORAGE_HOSTrrrr_get_default_storage_base_urlUsrcCstttkrtSdS)zIThis is an experimental configuration variable. Use api_endpoint instead.N)rrrrrrr_get_api_endpoint_override[srFcCs8t|}|d|j}t|j||r(dndddf}|S)zReturns the scheme and netloc sections of the url, with the bucket prepended to the netloc. Not intended for use with netlocs which include a username and password. ./)rnetlocrscheme)urlZbucketZtrailing_slash parsed_urlZ new_netlocbase_urlrrr_virtual_hosted_style_base_urlbs r&cCstddkS)NZ!GOOGLE_API_USE_CLIENT_CERTIFICATEtrue)rrrrrr_use_client_certpsr(cCsttjttjSr)rrrZPROJECTZLEGACY_PROJECTrrrr_get_environ_projectts r)cCs4|dkr dSt|d|dgs0td|S)zPre-flight ``Bucket`` name validation. :type name: str or :data:`NoneType` :param name: Proposed bucket name. :rtype: str or :data:`NoneType` :returns: ``name`` if valid. Nrz8Bucket names must start and end with a number or letter.)allisalnum ValueError)namerrr_validate_name{s r/c @seZdZdZdddZeddZeddZed d Zd d Z d dZ eddZ dddddddde e df ddZddZddZddddde e dfddZddddde edfddZdS)_PropertyMixina1Abstract mixin for cloud storage classes with associated properties. Non-abstract subclasses should implement: - path - client - user_project :type name: str :param name: The name of the object. Bucket names must start and end with a number or letter. NcCs||_i|_t|_dSr)r. _propertiesset_changes)selfr.rrr__init__sz_PropertyMixin.__init__cCstdS)z$Abstract getter for the object path.NNotImplementedErrorr4rrrpathsz_PropertyMixin.pathcCstdS)z&Abstract getter for the object client.Nr6r8rrrclientsz_PropertyMixin.clientcCstdS)z,Abstract getter for the object user_project.Nr6r8rrr user_projectsz_PropertyMixin.user_projectcCs|dkr|j}|S)aCheck client or verify over-ride. :type client: :class:`~google.cloud.storage.client.Client` or ``NoneType`` :param client: the client to use. If not passed, falls back to the ``client`` stored on the current object. :rtype: :class:`google.cloud.storage.client.Client` :returns: The client passed in or the currently bound client. N)r:)r4r:rrr_require_clients z_PropertyMixin._require_clientcCsiS)a@Return any encryption headers needed to fetch the object. .. note:: Defined here because :meth:`reload` calls it, but this method is really only relevant for :class:`~google.cloud.storage.blob.Blob`. :rtype: dict :returns: a mapping of encryption-related headers. rr8rrr_encryption_headerss z"_PropertyMixin._encryption_headerscCsi}|jdk r|j|d<|S)zDefault query parameters.NZ userProject)r;)r4paramsrrr _query_paramss  z_PropertyMixin._query_paramsZnoAclc Cs||}|j} || d<t| ||||d| dk rD| | d<|j| d<|} t| ||d|j|j| | | | |d}||dS)aReload properties from Cloud Storage. If :attr:`user_project` is set, bills the API request to that project. :type client: :class:`~google.cloud.storage.client.Client` or ``NoneType`` :param client: the client to use. If not passed, falls back to the ``client`` stored on the current object. :type projection: str :param projection: (Optional) If used, must be 'full' or 'noAcl'. Defaults to ``'noAcl'``. Specifies the set of properties to return. :type if_etag_match: Union[str, Set[str]] :param if_etag_match: (Optional) See :ref:`using-if-etag-match` :type if_etag_not_match: Union[str, Set[str]]) :param if_etag_not_match: (Optional) See :ref:`using-if-etag-not-match` :type if_generation_match: long :param if_generation_match: (Optional) See :ref:`using-if-generation-match` :type if_generation_not_match: long :param if_generation_not_match: (Optional) See :ref:`using-if-generation-not-match` :type if_metageneration_match: long :param if_metageneration_match: (Optional) See :ref:`using-if-metageneration-match` :type if_metageneration_not_match: long :param if_metageneration_not_match: (Optional) See :ref:`using-if-metageneration-not-match` :type timeout: float or tuple :param timeout: (Optional) The amount of time, in seconds, to wait for the server response. See: :ref:`configuring_timeouts` :type retry: google.api_core.retry.Retry or google.cloud.storage.retry.ConditionalRetryPolicy :param retry: (Optional) How to retry the RPC. See: :ref:`configuring_retries` :type soft_deleted: bool :param soft_deleted: (Optional) If True, looks for a soft-deleted object. Will only return the object metadata if the object exists and is in a soft-deleted state. :attr:`generation` is required to be set on the blob if ``soft_deleted`` is set to True. See: https://cloud.google.com/storage/docs/soft-delete projectionr r rrNZ softDeleted generation)r r ) query_paramsheaderstimeoutretry_target_object) r<r? _add_generation_match_parametersrBr=_add_etag_match_headersZ _get_resourcer9_set_properties)r4r:r@r r r r rrrErFZ soft_deletedrCrD api_responserrrreloads8B  z_PropertyMixin.reloadcCs|j|||j|<dS)aUpdate field of this object's properties. This method will only update the field provided and will not touch the other fields. It **will not** reload the properties from the server. The behavior is local only and syncing occurs via :meth:`patch`. :type name: str :param name: The field name to update. :type value: object :param value: The value being updated. N)r3addr1)r4r.valuerrr_patch_property2s z_PropertyMixin._patch_propertycCs||_t|_dS)zSet the properties for the current object. :type value: dict or :class:`google.cloud.storage.batch._FutureDict` :param value: The properties to be set. N)r1r2r3r4rNrrrrJDsz_PropertyMixin._set_propertiesFc sp|}j} d| d<|r$|| d<t| ||||dfddjD} |jj| | ||d} | dS) ajSends all changed properties in a PATCH request. Updates the ``_properties`` with the response from the backend. If :attr:`user_project` is set, bills the API request to that project. :type client: :class:`~google.cloud.storage.client.Client` or ``NoneType`` :param client: the client to use. If not passed, falls back to the ``client`` stored on the current object. :type if_generation_match: long :param if_generation_match: (Optional) See :ref:`using-if-generation-match` :type if_generation_not_match: long :param if_generation_not_match: (Optional) See :ref:`using-if-generation-not-match` :type if_metageneration_match: long :param if_metageneration_match: (Optional) See :ref:`using-if-metageneration-match` :type if_metageneration_not_match: long :param if_metageneration_not_match: (Optional) See :ref:`using-if-metageneration-not-match` :type timeout: float or tuple :param timeout: (Optional) The amount of time, in seconds, to wait for the server response. See: :ref:`configuring_timeouts` :type retry: google.api_core.retry.Retry or google.cloud.storage.retry.ConditionalRetryPolicy :param retry: (Optional) How to retry the RPC. See: :ref:`configuring_retries` :type override_unlocked_retention: bool :param override_unlocked_retention: (Optional) override_unlocked_retention must be set to True if the operation includes a retention property that changes the mode from Unlocked to Locked, reduces the retainUntilTime, or removes the retention configuration from the object. See: https://cloud.google.com/storage/docs/json_api/v1/objects/patch fullr@overrideUnlockedRetentionrAcsi|]}|j|qSr)r1).0keyr8rr sz(_PropertyMixin.patch..)rCrGrErFN)r<r?rHr3Z_patch_resourcer9rJ) r4r:r r rrrErFoverride_unlocked_retentionrCZupdate_propertiesrKrr8rpatchNs,6 z_PropertyMixin.patchc Cs^||}|j} d| d<|r$|| d<t| ||||d|j|j|j| |||d} || dS)a`Sends all properties in a PUT request. Updates the ``_properties`` with the response from the backend. If :attr:`user_project` is set, bills the API request to that project. :type client: :class:`~google.cloud.storage.client.Client` or ``NoneType`` :param client: the client to use. If not passed, falls back to the ``client`` stored on the current object. :type if_generation_match: long :param if_generation_match: (Optional) See :ref:`using-if-generation-match` :type if_generation_not_match: long :param if_generation_not_match: (Optional) See :ref:`using-if-generation-not-match` :type if_metageneration_match: long :param if_metageneration_match: (Optional) See :ref:`using-if-metageneration-match` :type if_metageneration_not_match: long :param if_metageneration_not_match: (Optional) See :ref:`using-if-metageneration-not-match` :type timeout: float or tuple :param timeout: (Optional) The amount of time, in seconds, to wait for the server response. See: :ref:`configuring_timeouts` :type retry: google.api_core.retry.Retry or google.cloud.storage.retry.ConditionalRetryPolicy :param retry: (Optional) How to retry the RPC. See: :ref:`configuring_retries` :type override_unlocked_retention: bool :param override_unlocked_retention: (Optional) override_unlocked_retention must be set to True if the operation includes a retention property that changes the mode from Unlocked to Locked, reduces the retainUntilTime, or removes the retention configuration from the object. See: https://cloud.google.com/storage/docs/json_api/v1/objects/patch rQr@rRrA)rCrErFrGN)r<r?rHZ _put_resourcer9r1rJ) r4r:r r rrrErFrVrCrKrrrupdates*6 z_PropertyMixin.update)N)__name__ __module__ __qualname____doc__r5propertyr9r:r;r<r=r?rrrLrOrJrWr rXrrrrr0sX       a  Sr0cs"fdd}fdd}t||S)zHCreate a property descriptor around the :class:`_PropertyMixin` helpers.cs |jS)zScalar property getter.)r1rr8 fieldnamerr_gettersz!_scalar_property.._gettercs||dS)zScalar property setter.N)rOrPr^rr_settersz!_scalar_property.._setter)r])r_r`rarr^r_scalar_propertys  rb cCs0||}t|dkr,||||}q dS)aRead blocks from a buffer and update a hash with them. :type buffer_object: bytes buffer :param buffer_object: Buffer containing bytes used to update a hash object. :type hash_obj: object that implements update :param hash_obj: A hash object (MD5 or CRC32-C). :type digest_block_size: int :param digest_block_size: The block size to write to the hash. Defaults to 8192. rN)readlenrX) buffer_objecthash_objZdigest_block_sizeblockrrr_write_buffer_to_hashs   ricCs"t}t|||}t|S)a Get MD5 hash of bytes (as base64). :type buffer_object: bytes buffer :param buffer_object: Buffer containing bytes used to compute an MD5 hash (as base64). :rtype: str :returns: A base64 encoded digest of the MD5 hash. )rridigestbase64 b64encode)rfrgZ digest_bytesrrr_base64_md5hashs  rmcKsBtD]8\}}||}|dk rt|tr.|g}d|||<qdS)zAdd generation match parameters into the given parameters list. :type headers: dict :param headers: Headers dict. :type match_parameters: dict :param match_parameters: if*etag*match parameters to add. N, )_ETAG_MATCH_PARAMETERSr isinstancestrjoin)rDmatch_parameterssnakecase_name header_namerNrrrrI!s   rIcKsZtD]P\}}||}|dk rt|tr8|||fqt|trL|||<qtdqdS)abAdd generation match parameters into the given parameters list. :type parameters: list or dict :param parameters: Parameters list or dict. :type match_parameters: dict :param match_parameters: if*generation*match parameters to add. :raises: :exc:`ValueError` if ``parameters`` is not a ``list()`` or a ``dict()``. Nz5`parameters` argument should be a dict() or a list().)_GENERATION_MATCH_PARAMETERSrrplistappenddictr-) parametersrsrtZcamelcase_namerNrrrrH3s     rHcKsbtdd|Ddkr^dd|D}d|dd}|d}d |d |}t|dS) zRaise ``ValueError`` exception if more than one parameter was set. :type error: :exc:`ValueError` :param error: Description of which fields were set :raises: :class:`~ValueError` containing the fields that were set css|]}|dk VqdSrr)rSargrrr Wsz._raise_if_more_than_one_set..cSsg|]}d|dqS)'r)rSr.rrr Xsz/_raise_if_more_than_one_set..rnNr*zPass at most one of z and )sumvalueskeysrrr-)kwargsZ escaped_keysZ keys_but_lastlast_keymsgrrr_raise_if_more_than_one_setOs rcCs&t|}|jr|jr|S|d|S)a!Helper to build bucket bound hostname URL. :type host: str :param host: Host name. :type scheme: str :param scheme: (Optional) Web scheme. If passed, use it as a scheme in the result URL. :rtype: str :returns: A bucket bound hostname URL. z://)rr"r!)hostr"Z url_partsrrr_bucket_bound_hostname_urlbs  rcCsdttS)Nzgccl-invocation-id/)rqrrrrr_get_invocation_idvsrapplication/json; charset=UTF-8cCs8|dt}|r"|d|7}dd||||p2|dS)aGet the headers for a request. :type user_agent: str :param user_agent: The user-agent for requests. :type command: str :param command: (Optional) Information about which interface for the operation was used, to be included in the X-Goog-API-Client header. Please leave as None unless otherwise directed. :rtype: dict :returns: The headers to be used for the request.  z gccl-gcs-cmd/zapplication/jsonz gzip, deflate)AcceptzAccept-Encodingz User-AgentzX-Goog-API-Clientz content-typezx-upload-content-type)r) user_agent content_typeZx_upload_content_typecommandZx_goog_api_clientrrr_get_default_headerszsr)F)rc)N)rNN)5r\rkdatetimehashlibrr urllib.parserruuidrZ google.authrZgoogle.cloud.storage.constantsrZgoogle.cloud.storage.retryrr rrZ_API_VERSION_OVERRIDE_ENV_VARZ_DEFAULT_UNIVERSE_DOMAINZ_STORAGE_HOST_TEMPLATEformatrrrZ _API_VERSIONrorvnowZ_NOWtimezoneutcZ_UTCrrrr&r(r)r/objectr0rbrirmrIrHrrrrrrrrs^          c