airflow.providers.amazon.aws.utils¶

Submodules¶

Attributes¶

`log`
`RESOURCE_IN_USE_RETRY_TIMEOUT`
`RESOURCE_IN_USE_RETRY_MAX_WAIT`

Functions¶

`is_resource_in_use_error`(exception)	Return True if the exception is a transient AWS `ResourceInUseException`.
`build_resource_in_use_retry_args`(logger)	Build tenacity arguments for retrying a call on a transient `ResourceInUseException`.
`trim_none_values`(obj)
`datetime_to_epoch`(date_time)	Convert a datetime object to an epoch integer (seconds).
`datetime_to_epoch_ms`(date_time)	Convert a datetime object to an epoch integer (milliseconds).
`datetime_to_epoch_utc_ms`(date_time)	Convert a datetime object to an epoch integer (milliseconds) in UTC timezone.
`datetime_to_epoch_us`(date_time)	Convert a datetime object to an epoch integer (microseconds).
`get_airflow_version`()
`get_botocore_version`()	Return the version number of the installed botocore package in the form of a tuple[int,...].
`validate_execute_complete_event`([event])
`validate_destination_path`(destination, base_path, *, ...)	Ensure `destination` stays within `base_path` once resolved.

Package Contents¶

airflow.providers.amazon.aws.utils.log[source]¶

airflow.providers.amazon.aws.utils.RESOURCE_IN_USE_RETRY_TIMEOUT = 300[source]¶

airflow.providers.amazon.aws.utils.RESOURCE_IN_USE_RETRY_MAX_WAIT = 60[source]¶

airflow.providers.amazon.aws.utils.is_resource_in_use_error(exception)[source]¶

Return True if the exception is a transient AWS ResourceInUseException.

airflow.providers.amazon.aws.utils.build_resource_in_use_retry_args(logger)[source]¶

Build tenacity arguments for retrying a call on a transient ResourceInUseException.

Shared by synchronous operators (tenacity.Retrying) and deferrable triggers (tenacity.AsyncRetrying) so both back off identically. reraise=True keeps the original error as the task failure once the retry timeout is exhausted. Accepts either an Airflow structlog logger (self.log) or a stdlib logging.Logger (module-level helpers).

airflow.providers.amazon.aws.utils.trim_none_values(obj)[source]¶

airflow.providers.amazon.aws.utils.datetime_to_epoch(date_time)[source]¶

Convert a datetime object to an epoch integer (seconds).

airflow.providers.amazon.aws.utils.datetime_to_epoch_ms(date_time)[source]¶

Convert a datetime object to an epoch integer (milliseconds).

airflow.providers.amazon.aws.utils.datetime_to_epoch_utc_ms(date_time)[source]¶

Convert a datetime object to an epoch integer (milliseconds) in UTC timezone.

airflow.providers.amazon.aws.utils.datetime_to_epoch_us(date_time)[source]¶

Convert a datetime object to an epoch integer (microseconds).

airflow.providers.amazon.aws.utils.get_airflow_version()[source]¶

airflow.providers.amazon.aws.utils.get_botocore_version()[source]¶

Return the version number of the installed botocore package in the form of a tuple[int,…].

airflow.providers.amazon.aws.utils.validate_execute_complete_event(event=None)[source]¶

airflow.providers.amazon.aws.utils.validate_destination_path(destination, base_path, *, base_name)[source]¶

Ensure destination stays within base_path once resolved.

In multi-file transfers the destination is base_path concatenated with a key suffix returned by list_keys. S3 object names are arbitrary strings controlled by whoever can write to the source bucket, so .. segments or an absolute name could place the upload outside base_path once the remote server resolves it on its host. base_name is the operator argument name used in the error message (e.g. sftp_path or ftp_path).