CloudWanderer

Main cloudwanderer module.

class CloudWanderer(storage_connectors, cloud_interface=None)

CloudWanderer.

Parameters
Return type

None

__init__(storage_connectors, cloud_interface=None)

Initialise CloudWanderer.

Parameters
Return type

None

write_resource(urn, service_resource_type_filters=None)

Fetch data for and persist to storage a single resource and its subresources.

If the resource does not exist it will be deleted from the storage connectors.

Example

Fetch a specific VPC and write it to a local Gremlin database.

>>> from cloudwanderer import CloudWanderer, ServiceResourceType, URN
>>> from cloudwanderer.storage_connectors import GremlinStorageConnector
>>> cloud_wanderer = CloudWanderer(storage_connectors=[
...        GremlinStorageConnector(
...          endpoint_url="ws://localhost:8182",
...        )
...    ])
>>> cloud_wanderer.write_resource(
...     urn=URN.from_string("urn:aws:123456789012:us-east-1:ec2:vpc:vpc-11111111")
... )
Parameters
Return type

None

write_resources(regions=None, service_resource_types=None, service_resource_type_filters=None)

Fetch all resources in this account from all regions and all services and write to storage.

All arguments are optional.

Example

Fetch AWS EC2 VPCs and write to a local Gremlin database.

>>> from cloudwanderer import CloudWanderer, ServiceResourceType
>>> from cloudwanderer.storage_connectors import GremlinStorageConnector
>>> cloud_wanderer = CloudWanderer(storage_connectors=[
...        GremlinStorageConnector(
...          endpoint_url="ws://localhost:8182",
...        )
...    ])
>>> cloud_wanderer.write_resources(
...     service_resource_types=[ServiceResourceType("ec2","vpc")]
... )
Parameters
Raises

ValueError – If invalid get/delete urns are produced by the cloud interface’s get_resource_discovery_actions

Return type

None

write_resources_concurrently(cloud_interface_generator, storage_connector_generator, concurrency=10, **kwargs)

Write all resources in this account from all regions and all services to storage.

Any additional args will be passed into the cloud interface’s get_ methods. WARNING: Experimental.

Parameters

  • concurrency (int) – Number of query threads to invoke concurrently.

  • cloud_interface_generator (Callable) – A method which returns a new cloud interface session when called. This helps prevent non-threadsafe cloud interfaces from interfering with each others.

  • storage_connector_generator (Callable) – A method which returns a list of storage connectors when called. The returned connectors should be instances of the same connectors each time the method is called. These connectors do not need to be thread safe and will be returned at the end of execution.

  • **kwargs – Additional keyword arguments will be passed down to the cloud interface methods.

Return type

List[cloudwanderer.cloud_wanderer.CloudWandererConcurrentWriteThreadResult]

class CloudWandererConcurrentWriteThreadResult(storage_connectors)

The result from write_resources_concurrently.

Parameters

storage_connectors (cloudwanderer.storage_connectors.base_connector.BaseStorageConnector) –

property storage_connectors

Alias for field number 0