Add support for endpoint_url for local dynamodb table

[vaibhavkhurana2018]

We will like to use credstash with the local dynamodb table created using https://github.com/localstack/localstack.

This PR adds support for adding endpoint_url while making the session to connect to the local dynamodb table rather than the remote AWS service.

It accepts the endpoint_url as an argument or via an environment variable DYNAMODB_ENDPOINT_URL, defaulting to None.

Have tested all the functions both with and without the endpoint_url. This will be a non-breaking change.

[vaibhavkhurana2018]

Create Table:

./credstash.py -t testing -r us-east-1 --endpoint_url http://localhost:4566 setup
Creating table...
Waiting for table to be created...
Adding tags...
Table has been created. Go read the README about how to create your KMS key

Put:

./credstash.py -t testing -r us-east-1 --endpoint_url http://localhost:4566 put test 'test'
test has been stored

List:

./credstash.py -t testing -r us-east-1 --endpoint_url http://localhost:4566 list
test -- version 0000000000000000001 -- comment

Get:

./credstash.py -t testing -r us-east-1 --endpoint_url http://localhost:4566 get test
test

GetAll:

./credstash.py -t testing -r us-east-1 --endpoint_url http://localhost:4566 getall
{
    "test": "test"
}

Localstack Running on Local:

CONTAINER ID        IMAGE                   COMMAND                  CREATED             STATUS              PORTS                                                                              NAMES
449fb6534869        localstack/localstack   "docker-entrypoint.sh"   About an hour ago   Up About an hour    0.0.0.0:4566->4566/tcp, 0.0.0.0:4571->4571/tcp, 0.0.0.0:8080-8081->8080-8081/tcp   localstack_main

[SamCullin]

Would love for this to go through

[SamCullin]

Been using this on my local for the past day and works well

[vaibhavkhurana2018]

Thanks @SamCullin !!

[jason-fugue]

Thanks for the PR! This is an awesome idea. I tried this out with localstack and it works great. I'm going to look into doing the same for KMS for offline testing.

I left one very minor suggestion about the new arg. Could you please update when you have time?

[jason-fugue]

Super-small issue - could you please change this to --endpoint-url to be consistent with the other args?

Suggested change

	parsers['super'].add_argument("--endpoint_url", default=os.environ.get("DYNAMODB_ENDPOINT_URL", None),
	parsers['super'].add_argument("--endpoint-url", default=os.environ.get("DYNAMODB_ENDPOINT_URL", None),

[vaibhavkhurana2018]

Done 👍

[SamCullin]

One thing that may be an issue is that people may expect the --endpoint-url to also update the KMS endpoint. Maybe something like --dynamo-endpoint-url might be more appropriate. So then if someone wants to add --kms-endpoint-url in the future they can.

[jason-fugue]

One thing that may be an issue is that people may expect the --endpoint-url to also update the KMS endpoint. Maybe something like --dynamo-endpoint-url might be more appropriate. So then if someone wants to add --kms-endpoint-url in the future they can.

This is a good point - and matching the env variable name would also be intuitive --dynamodb-endpoint-url. @vaibhavkhurana2018 I apologize that your PR has sat for so long already, but if this is something that you could do it would be appreciated. Otherwise, I can take care of it after the merge.

I'll create a followup issue to add similar support for the KMS endpoint.

[vertig0ne]

This PR was needed for a use case I had, I had to make further changes to get it working, one typographic error and also add in KMS to the endpoint_url.

diff --git a/credstash.py b/credstash.py
index aa8735b..c78a555 100755
--- a/credstash.py
+++ b/credstash.py
@@ -329,7 +329,7 @@ def putSecret(name, secret, version="", kms_key="alias/credstash",
         if dynamodb is None:
             dynamodb = session.resource('dynamodb', region_name=region, endpoint_url=endpoint_url)
         if kms is None:
-            kms = session.client('kms', region_name=kms_region or region)
+            kms = session.client('kms', region_name=kms_region or region, endpoint_url=endpoint_url)
 
     key_service = KeyService(kms, kms_key, context)
     sealed = seal_aes_ctr_legacy(
@@ -565,7 +565,7 @@ def getSecret(name, version="", region=None, endpoint_url=None, table="credentia
         if dynamodb is None:
             dynamodb = session.resource('dynamodb', region_name=region, endpoint_url=endpoint_url)
         if kms is None:
-            kms = session.client('kms', region_name=kms_region or region)
+            kms = session.client('kms', region_name=kms_region or region, endpoint_url=endpoint_url)
 
     secrets = dynamodb.Table(table)
 
@@ -1112,7 +1112,7 @@ def main():
     # test for region
     try:
         region = args.region
-        endpoint_url = args.endpoint-url
+        endpoint_url = args.endpoint_url
         session = get_session(**session_params)
         session.resource('dynamodb', region_name=region, endpoint_url=endpoint_url)
     except botocore.exceptions.NoRegionError: