CloudWanderer

Main cloudwanderer module.

class CloudWanderer(storage_connectors, boto3_session=None)

CloudWanderer.

Parameters
property account_id

Return the AWS Account ID our boto3 session is authenticated against.

property enabled_regions

Return a list of enabled regions in this account.

write_resources(exclude_resources=None, client_args=None)

Write all AWS resources in this account from all regions and all services to storage.

Parameters
  • exclude_resources (list) – A list of resource names to exclude (e.g. ['instance'])

  • client_args (dict) – Arguments to pass into the boto3 client. See: boto3.session.Session.client()

Return type

None

write_resources_concurrently(session_generator, exclude_resources=None, client_args=None, concurrency=10)

Write all AWS resources in this account from all regions and all services to storage.

Parameters
  • exclude_resources (list) – A list of resource names to exclude (e.g. ['instance'])

  • client_args (dict) – Arguments to pass into the boto3 client. See: boto3.session.Session.client()

  • concurrency (int) – Number of query threads to invoke concurrently. If the number of threads exceeds the number of regions by at least two times multiple services to be queried concurrently in each region. WARNING: Experimental. Complete data capture depends heavily on the thread safeness of the storage connector and has not been thoroughly tested!

  • session_generator (callable) – A method which returns a new boto3 session when called. You’re usually safe to pass in boto3.session.Session

Return type

None

write_resources_in_region(exclude_resources=None, region_name=None, client_args=None)

Write all AWS resources in this account region from all services to storage.

Parameters
  • exclude_resources (list) – A list of resource names to exclude (e.g. ['instance'])

  • region_name (str) – The name of the region to get resources from (defaults to session default if not specified)

  • client_args (dict) – Arguments to pass into the boto3 client. See: boto3.session.Session.client()

Return type

None

write_resources_of_service_in_region(service_name, exclude_resources=None, region_name=None, client_args=None)

Write all AWS resources in this region in this service to storage.

Cleans up any resources in the StorageConnector that no longer exist.

Parameters
  • service_name (str) – The name of the service to write resources for (e.g. 'ec2')

  • exclude_resources (list) – A list of resource names to exclude (e.g. ['instance'])

  • region_name (str) – The name of the region to get resources from (defaults to session default if not specified)

  • client_args (dict) – Arguments to pass into the boto3 client. See: boto3.session.Session.client()

Return type

None

write_resources_of_type_in_region(service_name, resource_type=None, region_name=None, client_args=None)

Write all AWS resources in this region in this service to storage.

Cleans up any resources in the StorageConnector that no longer exist.

Parameters
  • service_name (str) – The name of the service to write resources for (e.g. 'ec2')

  • resource_type (str) – The name of the type of the resource to write (e.g. 'instance')

  • region_name (str) – The name of the region to get resources from (defaults to session default if not specified)

  • client_args (dict) – Arguments to pass into the boto3 client. See: boto3.session.Session.client()

Return type

None

write_secondary_attributes(exclude_resources=None, client_args=None)

Write all secondary attributes in this account from all regions and all services to storage.

Parameters
  • exclude_resources (list) – A list of resource names to exclude (e.g. ['instance'])

  • client_args (dict) – Arguments to pass into the boto3 client. See: boto3.session.Session.client()

Return type

None

write_secondary_attributes_in_region(exclude_resources=None, region_name=None, client_args=None)

Write all secondary attributes in this account in this region to storage.

These custom resource attribute definitions allow us to fetch secondary attributes that are not returned by the resource’s default describe calls. Unlike write_resources() and write_resources_of_type_in_region() this method does not clean up stale secondary attributes from storage.

Parameters
  • exclude_resources (list) – A list of resources not to write attributes for (e.g. ['vpc'])

  • region_name (str) – The name of the region to get resources from (defaults to session default if not specified)

  • client_args (dict) – Arguments to pass into the boto3 client. See: boto3.session.Session.client()

Return type

None

write_secondary_attributes_of_service_in_region(service_name, exclude_resources=None, region_name=None, client_args=None)

Write all secondary attributes in this account in this service to storage.

These custom resource attribute definitions allow us to fetch secondary attributes that are not returned by the resource’s default describe calls. Unlike write_resources() and write_resources_of_type_in_region() this method does not clean up stale secondary attributes from storage.

Parameters
  • service_name (str) – The name of the service to write the attributes of (e.g. 'ec2')

  • exclude_resources (list) – A list of resources not to write attributes for (e.g. ['vpc'])

  • region_name (str) – The name of the region to get resources from (defaults to session default if not specified)

  • client_args (dict) – Arguments to pass into the boto3 client. See: boto3.session.Session.client()

Return type

None

write_secondary_attributes_of_type_in_region(service_name, resource_type, region_name=None, client_args=None)

Write all secondary attributes in this account of this resource type to storage.

These custom resource attribute definitions allow us to fetch secondary attributes that are not returned by the resource’s default describe calls.

Parameters
  • service_name (str) – The name of the service to write the attributes of (e.g. ec2)

  • resource_type (str) – The type of resource to write the attributes of (e.g. instance)

  • region_name (str) – The name of the region to get resources from (defaults to session default if not specified)

  • client_args (dict) – Arguments to pass into the boto3 client. See: boto3.session.Session.client()

Return type

None