o  i\@sdZddlZddlZddlmZddlZddlZddlZddlm Z ddlm Z ddl m Z ddl mZddlmZdd lmZdd lmZd Z d Z d Z dZdZejedZdZeedZ dZdZejj Z!ej"j#Z$ddZ%ddZ&ddZ'dd3d4Z4d5d6Z5 7  d?d8d9Z6d:d;Z7dS)@zVHelper functions for Cloud Storage utility classes. These are *not* part of the API. N)md5)urlsplit) urlunsplit)uuid4)environment_vars)_DEFAULT_TIMEOUT) DEFAULT_RETRY))DEFAULT_RETRY_IF_METAGENERATION_SPECIFIEDSTORAGE_EMULATOR_HOSTAPI_ENDPOINT_OVERRIDEAPI_VERSION_OVERRIDEzgoogleapis.comzstorage.{universe_domain})universe_domainzhttps://v1)) if_etag_matchzIf-Match)if_etag_not_matchz If-None-Match))if_generation_matchifGenerationMatch)if_generation_not_matchifGenerationNotMatch)if_metageneration_matchifMetagenerationMatch)if_metageneration_not_matchifMetagenerationNotMatch)if_source_generation_matchifSourceGenerationMatch)if_source_generation_not_matchifSourceGenerationNotMatch)if_source_metageneration_matchifSourceMetagenerationMatch)"if_source_metageneration_not_matchifSourceMetagenerationNotMatchcCstjtdSN)osenvirongetSTORAGE_EMULATOR_ENV_VARr&r&}/var/www/snowflake_co_dev_github/snow_flake_back_end_deploy/env/lib/python3.10/site-packages/google/cloud/storage/_helpers.py_get_storage_emulator_overrideSr(cCsttttSr!)r"getenv_API_ENDPOINT_OVERRIDE_ENV_VAR_DEFAULT_SCHEME_TRUE_DEFAULT_STORAGE_HOSTr&r&r&r'_get_default_storage_base_urlWsr.cCstttkr tSdS)zIThis is an experimental configuration variable. Use api_endpoint instead.N)r.r,r-r&r&r&r'_get_api_endpoint_override]sr/FcCs8t|}|d|j}t|j||rdndddf}|S)zReturns the scheme and netloc sections of the url, with the bucket prepended to the netloc. Not intended for use with netlocs which include a username and password. ./)rnetlocrscheme)urlbuckettrailing_slash parsed_url new_netlocbase_urlr&r&r'_virtual_hosted_style_base_urlds r;cCstddkS)N!GOOGLE_API_USE_CLIENT_CERTIFICATEtrue)r"r*r&r&r&r'_use_client_certrr)r>cCsttjttjSr!)r"r*rPROJECTLEGACY_PROJECTr&r&r&r'_get_environ_projectvs rAcCs4|durdSt|d|dgstd|S)zPre-flight ``Bucket`` name validation. :type name: str or :data:`NoneType` :param name: Proposed bucket name. :rtype: str or :data:`NoneType` :returns: ``name`` if valid. Nrz8Bucket names must start and end with a number or letter.)allisalnum ValueError)namer&r&r'_validate_name}s  rGc @seZdZdZdddZeddZeddZed d Zd d Z d dZ eddZ dddddddde e df ddZddZddZddddde e dfddZddddde edfddZdS)_PropertyMixina1Abstract mixin for cloud storage classes with associated properties. Non-abstract subclasses should implement: - path - client - user_project :type name: str :param name: The name of the object. Bucket names must start and end with a number or letter. NcCs||_i|_t|_dSr!)rF _propertiesset_changes)selfrFr&r&r'__init__s z_PropertyMixin.__init__cCt)z$Abstract getter for the object path.NotImplementedErrorrLr&r&r'pathz_PropertyMixin.pathcCrN)z&Abstract getter for the object client.rOrQr&r&r'clientrSz_PropertyMixin.clientcCrN)z,Abstract getter for the object user_project.rOrQr&r&r' user_projectrSz_PropertyMixin.user_projectcCs|dur|j}|S)aCheck client or verify over-ride. :type client: :class:`~google.cloud.storage.client.Client` or ``NoneType`` :param client: the client to use. If not passed, falls back to the ``client`` stored on the current object. :rtype: :class:`google.cloud.storage.client.Client` :returns: The client passed in or the currently bound client. N)rT)rLrTr&r&r'_require_clients z_PropertyMixin._require_clientcCsiS)a@Return any encryption headers needed to fetch the object. .. note:: Defined here because :meth:`reload` calls it, but this method is really only relevant for :class:`~google.cloud.storage.blob.Blob`. :rtype: dict :returns: a mapping of encryption-related headers. r&rQr&r&r'_encryption_headerss z"_PropertyMixin._encryption_headerscCsi}|jdur |j|d<|S)zDefault query parameters.N userProject)rU)rLparamsr&r&r' _query_paramss  z_PropertyMixin._query_paramsnoAclc Cs||}|j} || d<t| ||||d| dur"| | d<|j| d<|} t| ||d|j|j| | | | |d}||dS)aReload properties from Cloud Storage. If :attr:`user_project` is set, bills the API request to that project. :type client: :class:`~google.cloud.storage.client.Client` or ``NoneType`` :param client: the client to use. If not passed, falls back to the ``client`` stored on the current object. :type projection: str :param projection: (Optional) If used, must be 'full' or 'noAcl'. Defaults to ``'noAcl'``. Specifies the set of properties to return. :type if_etag_match: Union[str, Set[str]] :param if_etag_match: (Optional) See :ref:`using-if-etag-match` :type if_etag_not_match: Union[str, Set[str]]) :param if_etag_not_match: (Optional) See :ref:`using-if-etag-not-match` :type if_generation_match: long :param if_generation_match: (Optional) See :ref:`using-if-generation-match` :type if_generation_not_match: long :param if_generation_not_match: (Optional) See :ref:`using-if-generation-not-match` :type if_metageneration_match: long :param if_metageneration_match: (Optional) See :ref:`using-if-metageneration-match` :type if_metageneration_not_match: long :param if_metageneration_not_match: (Optional) See :ref:`using-if-metageneration-not-match` :type timeout: float or tuple :param timeout: (Optional) The amount of time, in seconds, to wait for the server response. See: :ref:`configuring_timeouts` :type retry: google.api_core.retry.Retry or google.cloud.storage.retry.ConditionalRetryPolicy :param retry: (Optional) How to retry the RPC. See: :ref:`configuring_retries` :type soft_deleted: bool :param soft_deleted: (Optional) If True, looks for a soft-deleted object. Will only return the object metadata if the object exists and is in a soft-deleted state. :attr:`generation` is required to be set on the blob if ``soft_deleted`` is set to True. See: https://cloud.google.com/storage/docs/soft-delete projectionrrrrN softDeleted generation)rr) query_paramsheaderstimeoutretry_target_object) rVrZ _add_generation_match_parametersr_rW_add_etag_match_headers _get_resourcerR_set_properties)rLrTr\rrrrrrrbrc soft_deletedr`ra api_responser&r&r'reloads4 B z_PropertyMixin.reloadcCs|j|||j|<dS)aUpdate field of this object's properties. This method will only update the field provided and will not touch the other fields. It **will not** reload the properties from the server. The behavior is local only and syncing occurs via :meth:`patch`. :type name: str :param name: The field name to update. :type value: object :param value: The value being updated. N)rKaddrI)rLrFvaluer&r&r'_patch_property4s z_PropertyMixin._patch_propertycCs||_t|_dS)zSet the properties for the current object. :type value: dict or :class:`google.cloud.storage.batch._FutureDict` :param value: The properties to be set. N)rIrJrKrLrmr&r&r'rhFs z_PropertyMixin._set_propertiesFc sp|}j} d| d<|r|| d<t| ||||dfddjD} |jj| | ||d} | dS) ajSends all changed properties in a PATCH request. Updates the ``_properties`` with the response from the backend. If :attr:`user_project` is set, bills the API request to that project. :type client: :class:`~google.cloud.storage.client.Client` or ``NoneType`` :param client: the client to use. If not passed, falls back to the ``client`` stored on the current object. :type if_generation_match: long :param if_generation_match: (Optional) See :ref:`using-if-generation-match` :type if_generation_not_match: long :param if_generation_not_match: (Optional) See :ref:`using-if-generation-not-match` :type if_metageneration_match: long :param if_metageneration_match: (Optional) See :ref:`using-if-metageneration-match` :type if_metageneration_not_match: long :param if_metageneration_not_match: (Optional) See :ref:`using-if-metageneration-not-match` :type timeout: float or tuple :param timeout: (Optional) The amount of time, in seconds, to wait for the server response. See: :ref:`configuring_timeouts` :type retry: google.api_core.retry.Retry or google.cloud.storage.retry.ConditionalRetryPolicy :param retry: (Optional) How to retry the RPC. See: :ref:`configuring_retries` :type override_unlocked_retention: bool :param override_unlocked_retention: (Optional) override_unlocked_retention must be set to True if the operation includes a retention property that changes the mode from Unlocked to Locked, reduces the retainUntilTime, or removes the retention configuration from the object. See: https://cloud.google.com/storage/docs/json_api/v1/objects/patch fullr\overrideUnlockedRetentionr]csi|]}|j|qSr&)rI).0keyrQr&r' z(_PropertyMixin.patch..)r`rdrbrcN)rVrZrerK_patch_resourcerRrh) rLrTrrrrrbrcoverride_unlocked_retentionr`update_propertiesrjr&rQr'patchPs, 6z_PropertyMixin.patchc Cs^||}|j} d| d<|r|| d<t| ||||d|j|j|j| |||d} || dS)a`Sends all properties in a PUT request. Updates the ``_properties`` with the response from the backend. If :attr:`user_project` is set, bills the API request to that project. :type client: :class:`~google.cloud.storage.client.Client` or ``NoneType`` :param client: the client to use. If not passed, falls back to the ``client`` stored on the current object. :type if_generation_match: long :param if_generation_match: (Optional) See :ref:`using-if-generation-match` :type if_generation_not_match: long :param if_generation_not_match: (Optional) See :ref:`using-if-generation-not-match` :type if_metageneration_match: long :param if_metageneration_match: (Optional) See :ref:`using-if-metageneration-match` :type if_metageneration_not_match: long :param if_metageneration_not_match: (Optional) See :ref:`using-if-metageneration-not-match` :type timeout: float or tuple :param timeout: (Optional) The amount of time, in seconds, to wait for the server response. See: :ref:`configuring_timeouts` :type retry: google.api_core.retry.Retry or google.cloud.storage.retry.ConditionalRetryPolicy :param retry: (Optional) How to retry the RPC. See: :ref:`configuring_retries` :type override_unlocked_retention: bool :param override_unlocked_retention: (Optional) override_unlocked_retention must be set to True if the operation includes a retention property that changes the mode from Unlocked to Locked, reduces the retainUntilTime, or removes the retention configuration from the object. See: https://cloud.google.com/storage/docs/json_api/v1/objects/patch rpr\rqr])r`rbrcrdN)rVrZre _put_resourcerRrIrh) rLrTrrrrrbrcrwr`rjr&r&r'updates* 6z_PropertyMixin.updater!)__name__ __module__ __qualname____doc__rMpropertyrRrTrUrVrWrZrrrkrnrhryr r{r&r&r&r'rHsZ       a  SrHcs"fdd}fdd}t||S)zHCreate a property descriptor around the :class:`_PropertyMixin` helpers.cs |jS)zScalar property getter.)rIr$rQ fieldnamer&r'_getters z!_scalar_property.._gettercs||dS)zScalar property setter.N)rnrorr&r'_settersz!_scalar_property.._setter)r)rrrr&rr'_scalar_propertys   r cCs>||}t|dkr||||}t|dks dSdS)aRead blocks from a buffer and update a hash with them. :type buffer_object: bytes buffer :param buffer_object: Buffer containing bytes used to update a hash object. :type hash_obj: object that implements update :param hash_obj: A hash object (MD5 or CRC32-C). :type digest_block_size: int :param digest_block_size: The block size to write to the hash. Defaults to 8192. rN)readlenr{) buffer_objecthash_objdigest_block_sizeblockr&r&r'_write_buffer_to_hashs   rcCs8tjdkr tdd}nt}t|||}t|S)a Get MD5 hash of bytes (as base64). :type buffer_object: bytes buffer :param buffer_object: Buffer containing bytes used to compute an MD5 hash (as base64). :rtype: str :returns: A base64 encoded digest of the MD5 hash. ) F)usedforsecurity)sys version_inforrdigestbase64 b64encode)rr digest_bytesr&r&r'_base64_md5hashs   rcKsBtD]\}}||}|durt|tr|g}d|||<qdS)zAdd generation match parameters into the given parameters list. :type headers: dict :param headers: Headers dict. :type match_parameters: dict :param match_parameters: if*etag*match parameters to add. N, )_ETAG_MATCH_PARAMETERSr$ isinstancestrjoin)ramatch_parameterssnakecase_name header_namermr&r&r'rf&s  rfcKsZtD](\}}||}|dur*t|tr|||fqt|tr&|||<qtdqdS)abAdd generation match parameters into the given parameters list. :type parameters: list or dict :param parameters: Parameters list or dict. :type match_parameters: dict :param match_parameters: if*generation*match parameters to add. :raises: :exc:`ValueError` if ``parameters`` is not a ``list()`` or a ``dict()``. Nz5`parameters` argument should be a dict() or a list().)_GENERATION_MATCH_PARAMETERSr$rlistappenddictrE) parametersrrcamelcase_namermr&r&r're8s    recKsbtdd|Ddkr/dd|D}d|dd}|d}d |d |}t|dS) zRaise ``ValueError`` exception if more than one parameter was set. :type error: :exc:`ValueError` :param error: Description of which fields were set :raises: :class:`~ValueError` containing the fields that were set css|]}|duVqdSr!r&)rrargr&r&r' \sz._raise_if_more_than_one_set..cSsg|]}d|dqS)'r&)rrrFr&r&r' ]ruz/_raise_if_more_than_one_set..rNrBzPass at most one of z and )sumvalueskeysrrE)kwargs escaped_keys keys_but_lastlast_keymsgr&r&r'_raise_if_more_than_one_setTsrcCs&t|}|jr |jr |S|d|S)a!Helper to build bucket bound hostname URL. :type host: str :param host: Host name. :type scheme: str :param scheme: (Optional) Web scheme. If passed, use it as a scheme in the result URL. :rtype: str :returns: A bucket bound hostname URL. z://)rr4r3)hostr4 url_partsr&r&r'_bucket_bound_hostname_urlgs rcCsdttS)Nzgccl-invocation-id/)rrr&r&r&r'_get_invocation_id{r)rapplication/json; charset=UTF-8cCs8|dt}|r|d|7}dd||||p|dS)aGet the headers for a request. :type user_agent: str :param user_agent: The user-agent for requests. :type command: str :param command: (Optional) Information about which interface for the operation was used, to be included in the X-Goog-API-Client header. Please leave as None unless otherwise directed. :rtype: dict :returns: The headers to be used for the request.  z gccl-gcs-cmd/zapplication/jsonz gzip, deflate)AcceptzAccept-Encodingz User-AgentzX-Goog-API-Clientz content-typezx-upload-content-type)r) user_agent content_typex_upload_content_typecommandx_goog_api_clientr&r&r'_get_default_headerssrcCstd}t|dS)aGenerates a secure 56 bit random integer. If 64 bit int is used, sometimes the random int generated is greater than max positive value of signed 64 bit int which is 2^63 -1 causing overflow issues. :rtype: int :returns: A secure random 56 bit integer. big)secrets token_bytesint from_bytes) random_bytesr&r&r'generate_random_56_bit_integers r)F)rr!)rNN)8rrdatetimehashlibrr"rr urllib.parserruuidr google.authrgoogle.cloud.storage.constantsrgoogle.cloud.storage.retryrr r%r+_API_VERSION_OVERRIDE_ENV_VAR_DEFAULT_UNIVERSE_DOMAIN_STORAGE_HOST_TEMPLATEformatr-r,r* _API_VERSIONrrnow_NOWtimezoneutc_UTCr(r.r/r;r>rArGobjectrHrrrrfrerrrrrr&r&r&r'sn          c   #