Migrating to AWS CodeCommit

Hosting your code in AWS CodeCommit has several advantages, the main one being seamless integration with AWS CodeDeploy and AWS CodePipeline.

I use SourceTree as my repo tool of choice, with Git/Bitbucket as the back end.

If you have a team of many developers and want to slowly migrate your code to AWS CodeCommit Git repo, you can setup your SourceTree config to push to both repo’s.

1. You will need a SSH-2-RSA 2048 Public/Private keys, this is what AWS supports. So once you have generated/imported the keys to AWS, you can then import the same key to your gihub or bitbucket account. Then just add them to your pageant. Read Setting Up AWS CodeCommit

2. In AWS, when you import your SSH keys for a IAM User, it will give you a SSH Key ID. Write down this SSH Key ID and the password for it will be the private key password you generated with PuttyGen. Always use a password for your private key file.



3. In SourceTree, go to Tools/Options and set the private key to your AWS SSH Key. Remember we added this to Bitbucket and Git, so we can now use the AWS SSH Key/Pairs for both repositories.

SourceTree Private Key

SourceTree Private Key

The last part, is to configure your local repo to post to both repositories, until you happy with the migration.

4. In SourceTree, select your repository, and go to Repository/Repository Settings. Then add a new origin. It will be in this format: ssh://git-codecommit.us-east-1.amazonaws.com/v1/repos/MyCoolApp

5. When it prompts for a username and password, enter your SSH Key ID and SSH private key password

Source Tree Remote

Source Tree Remote

Once you happy with the migration, you can then set AWS CodeCommit as the default remote, by ticking the checkbox. You may need to first rename the original remote “origin” to “old” then set AWS as the default 🙂

My only gripe with CodeCommit is no built in hooks to deploy directly to S3.  This would be great for static assets.

#CodeCommit #AWS


Getting started with Amazon SQS

The data and metadata inflection point

We are nearing an inflection point regarding technology and data. Data is basically gold. In the next 30 years you will basically have a life logger app and many connected/smart devices. You will be able to rewind back in time and listen/look at a conversation you had with a random guy you met at a party.

You will punch into the Amazon Life Timeline service “Go to when a met a man wearing a shirt with Sponge Bob on it”. You wait a few seconds and a video appears at the exact moment you met the guy at a party.

Our lives will be transparent, our ego will be lowered, because we as a species are happy to share and be transparent. We will tip towards transparency and away from privacy, why? Because if we do not, we create a stumbling block in our technological evolution. Data is king, and this is the fundamental reason why companies pay so much for apps.

Google is not a search engine, it is going to be the most powerful artificial intelligent service offering in the world. One day you will use Google’s AI to optimize your life. It will track how you drive, when you sleep, when you come home; and by doing so, will have enough data collection points to run AI routines on your data and provide you with awesome benefits.

Likewise, Amazon Machine Learning services will be our AI friend.

As programmers we going to need to store data about data or data about the bits that we send to the internet… Metadata.

One way to do so is by asynchronous messaging. You will of course have an App/Smart Device that needs to send data or metadata about user behavior.

Queue Sender

Your app can send small data messages as a user is consuming your service to an Amazon Queue on the cloud.

using System;
using System.Collections;
using System.Collections.Generic;
using System.Web;
using Amazon;
using Amazon.Runtime;
using Amazon.SQS;
using Amazon.SQS.Model;
using Amazon.Util;

namespace Wangle.Queue.Client
    public class AwsClient : IQueueClient
        private AmazonSQSClient _client;
        private string defaultQueueUrl;

        public void Initialize(string url)
            ProfileManager.RegisterProfile("Wangle", "myaccessKey","mysecretkey");
            var amazonSqsConfig = new AmazonSQSConfig { ServiceURL = "http://sqs.us-east-1.amazonaws.com" };
            _client = new AmazonSQSClient(ProfileManager.GetAWSCredentials("Wangle"), amazonSqsConfig);
            defaultQueueUrl = url;

        public void SendMessage(string message)
            var sendMessageRequest = new SendMessageRequest
                QueueUrl = defaultQueueUrl,
                MessageBody = $"{message} + {DateTimeOffset.UtcNow}" //Unicode Only!



        public IList<string> ReceiveMessage()
            var data = new List<string>();
            var receiveMessageRequest = new ReceiveMessageRequest
                QueueUrl = defaultQueueUrl,
                MaxNumberOfMessages = 10,

            var receiveMessageResponse = _client.ReceiveMessage(receiveMessageRequest);

            receiveMessageResponse.Messages.ForEach(m =>
                var receiptHandle = m.ReceiptHandle;
                _client.DeleteMessageAsync(defaultQueueUrl, receiptHandle);
            return data;

Cloud Data Retention Receiver

Once the message is now in the message queue in the cloud, you will have a service in the cloud process this message and store it a Big Data Service. Below is the code to get the the message off the queue.

class Program
        static void Main(string[] args)
            //Fake a worker role service running in Amazon Cloud that processes data storage.
            Console.WriteLine("Fetching data logs from queue to prepare for governance...");
            var queueClient = new AwsClient();

            while (true)
                queueClient.ReceiveMessage().ToList().ForEach(m => Console.WriteLine(m));

                //ToDo: Store the audit data in AmazonS3 or Big Data service: User, Url, DateTimeUtc, SourceIP, DestIP




So that is basically the code you need. Of course you will need to install the Amazon Service SDK from Nuget.

This should get you going in the right direction when you need to send data to the cloud over the wire for later processing.

I am sure Amazon SQS will be used to start sending data asynchronously for  Fitbit information, how long you sleep for, how you drive your car and many more. Soon all our devices will be smart e.g. A cooking pot with a chip, your shirt with a chip …

See you soon in VR land…