Storage Connector Reference¶
DynamoDB Connector¶
Allows CloudWanderer to store resources in DynamoDB.
-
class
DynamoDbConnector
(table_name='cloud_wanderer', endpoint_url=None, boto3_session=None, client_args=None, number_of_shards=10)¶ CloudWanderer Storage Connector for DynamoDB.
- Parameters
table_name (str) – The name of the table to store resources in.
endpoint_url (str) – Optional override endpoint url for DynamoDB.
boto3_session (boto3.session.Session) – Optional boto3 session to use to interact with DynamoDB. Useful if your DynamoDB table is in a different account/region to your configured defaults.
number_of_shards (int) – The number of shards to break records across low-cardinality indices. Prevents hot-partitions. If you don’t know what this means, ignore this setting.
client_args (dict) – Arguments to pass into the boto3 client. See:
boto3.session.Session.client()
Example
>>> import cloudwanderer >>> cloud_wanderer = cloudwanderer.CloudWanderer( ... storage_connectors=[cloudwanderer.storage_connectors.DynamoDbConnector( ... endpoint_url='http://localhost:8000' ... )] ... )
-
delete_resource
(urn)¶ Delete the resource and all its resource attributes from DynamoDB.
- Parameters
urn (AwsUrn) – The URN of the resource to delete from Dynamo
- Return type
None
-
delete_resource_of_type_in_account_region
(service, resource_type, account_id, region, urns_to_keep=None)¶ Delete resources of type in account id unless in list of URNs.
- Parameters
account_id (str) – AWS Account ID
region (str) – AWS region (e.g.
'eu-west-2'
)service (str) – Service name (e.g.
'ec2'
)resource_type (str) – Resource Type (e.g.
'instance'
)urns_to_keep (List[cloudwanderer.aws_urn.AwsUrn]) – A list of resources not to delete
- Return type
None
-
init
()¶ Create the DynamoDB Database.
- Return type
None
-
read_all
()¶ Return raw data from all DynamoDB table records (not just resources).
- Return type
Iterator[dict]
-
read_resource
(urn)¶ Return the resource with the specified
cloudwanderer.aws_urn.AwsUrn
.- Parameters
urn (AwsUrn) – The AWS URN of the resource to return
- Return type
-
read_resources
(**kwargs)¶ Return the resources matching the arguments.
All arguments are optional, though some will fallback to performing a table scan.
- Parameters
urn (cloudwanderer.aws_urn.AwsUrn) – The AWS URN of the resource to return
account_id (str) – AWS Account ID
region (str) – AWS region (e.g.
'eu-west-2'
)service (str) – Service name (e.g.
'ec2'
)resource_type (str) – Resource Type (e.g.
'instance'
)
- Return type
Iterator[cloudwanderer.cloud_wanderer_resource.CloudWandererResource]
-
write_resource
(urn, resource)¶ Write the specified resource to DynamoDB.
- Parameters
urn (cloudwanderer.aws_urn.AwsUrn) – The URN of the resource.
resource (boto3.resources.base.ServiceResource) – The boto3 Resource object representing the resource.
- Return type
None
-
write_secondary_attribute
(urn, attribute_type, secondary_attribute)¶ Write the specified resource attribute to DynamoDb.
- Parameters
urn (AwsUrn) – The resource whose attribute to write.
attribute_type (str) – The type of the resource attribute to write (usually the boto3 client method name)
secondary_attribute (boto3.resources.base.ServiceResource) – The resource attribute to write to storage.
- Return type
None
Memory Connector¶
-
class
MemoryStorageConnector
(*args, **kwargs)¶ Storage connector to place data in memory.
Useful for testing.
Example
>>> import cloudwanderer >>> cloud_wanderer = cloudwanderer.CloudWanderer( ... storage_connectors=[cloudwanderer.storage_connectors.MemoryStorageConnector()] ... )
-
delete_resource
(urn)¶ Delete the resource and all its resource attributes from memory.
- Parameters
urn (cloudwanderer.aws_urn.AwsUrn) –
- Return type
None
-
delete_resource_of_type_in_account_region
(service, resource_type, account_id, region, urns_to_keep=None)¶ Delete resources of type in account id unless in list of URNs.
- Parameters
service (str) –
resource_type (str) –
account_id (str) –
region (str) –
urns_to_keep (Optional[List[cloudwanderer.aws_urn.AwsUrn]]) –
- Return type
None
-
init
()¶ Dummy method to fulfil interface requirements.
- Return type
None
-
read_all
()¶ Return the raw dictionaries stored in memory.
- Return type
Iterator[dict]
-
read_resource
(urn)¶ Return the resource with the specified
cloudwanderer.aws_urn.AwsUrn
.- Parameters
urn (cloudwanderer.aws_urn.AwsUrn) – The AWS URN of the resource to return
- Return type
-
read_resources
(**kwargs)¶ Return the resources matching the arguments.
All arguments are optional
- Parameters
urn (cloudwanderer.aws_urn.AwsUrn) – The AWS URN of the resource to return
account_id (str) – AWS Account ID
region (str) – AWS region (e.g.
'eu-west-2'
)service (str) – Service name (e.g.
'ec2'
)resource_type (str) – Resource Type (e.g.
'instance'
)
- Return type
Iterator[cloudwanderer.cloud_wanderer_resource.CloudWandererResource]
-
write_resource
(urn, resource)¶ Write the specified resource to memory.
- Parameters
urn (cloudwanderer.aws_urn.AwsUrn) – The URN of the resource.
resource (boto3.resources.base.ServiceResource) – The boto3 Resource object representing the resource.
- Return type
None
-
write_secondary_attribute
(urn, attribute_type, secondary_attribute)¶ Write the specified resource attribute to DynamoDb.
- Parameters
urn (AwsUrn) – The resource whose attribute to write.
attribute_type (str) – The type of the resource attribute to write (usually the boto3 client method name)
secondary_attribute (boto3.resources.base.ServiceResource) – The resource attribute to write to storage.
- Return type
None
-
Base Connector¶
Module containing abstract classes for CloudWanderer storage connectors.
-
class
BaseStorageConnector
¶ Abstract class for specification of the CloudWanderer storage connector interface.
-
abstract
delete_resource
(urn)¶ Delete this resource and all its resource attributes.
- Parameters
urn (cloudwanderer.aws_urn.AwsUrn) –
- Return type
None
-
abstract
delete_resource_of_type_in_account_region
(service, resource_type, account_id, region, urns_to_keep=None)¶ Delete resources of type in account and region unless in list of URNs.
This is used primarily to clean up old resources.
- Parameters
service (str) –
resource_type (str) –
account_id (str) –
region (str) –
urns_to_keep (Optional[List[cloudwanderer.aws_urn.AwsUrn]]) –
- Return type
None
-
abstract
init
()¶ Initialise the storage backend whatever it is.
- Return type
None
-
abstract
read_all
()¶ Return all records from storage.
- Return type
Iterator[dict]
-
abstract
read_resource
(urn)¶ Return a resource matching the supplied urn from storage.
- Parameters
urn (cloudwanderer.aws_urn.AwsUrn) –
- Return type
-
abstract
read_resources
(urn, account_id, region, service, resource_type)¶ Yield a resource matching the supplied urn from storage.
- Parameters
urn (cloudwanderer.aws_urn.AwsUrn) –
account_id (str) –
region (str) –
service (str) –
resource_type (str) –
- Return type
Iterator[cloudwanderer.cloud_wanderer_resource.CloudWandererResource]
-
abstract
write_resource
(urn, resource)¶ Persist a single resource to storage.
- Parameters
urn (cloudwanderer.aws_urn.AwsUrn) –
resource (str) –
- Return type
None
-
abstract
write_secondary_attribute
(urn, attribute_type, secondary_attribute)¶ Write the specified resource attribute to DynamoDb.
- Parameters
urn (AwsUrn) – The resource whose attribute to write.
attribute_type (str) – The type of the resource attribute to write (usually the boto3 client method name)
secondary_attribute (boto3.resources.base.ServiceResource) – The resource attribute to write to storage.
- Return type
None
-
abstract