CloudFormation – Template for S3, Route53 and CloudFront

So, I was working on a project for the client who happens to be a famous standup comedian here in London. He asked us to develop a WordPress based website where he can post about his tours, recent activities and videos.

We created a standalone website using AngularJS but exposed the JSON based API on WordPress which is fairly a good approach.

Lets talk about how I designed the frontend architecture of this website. As what we have seen that the requirement of this project is not uncommon and can easily be mapped to other clients, I decided to give it a shot by creating a AWS CloudFormation template which would utilize a couple of AWS resources mentioned down below.

1) S3 Bucket with website hosting enabled.

2) CloudFront Distribution with the newly created S3 Bucket as Origin.

3) A new Hosted Zone on Route53 for website domain.

4)  An Alias record for the hosted zone which maps to the CloudFront Distribution.

Design

CloudFormation-Template

Template File

The actual template in JSON has been uploaded on Github as well. Please feel free to improve it and also share your feedback.

Link: https://github.com/uzairjawed/CloudFormation-Template-With-Route53-CloudFront-S3

Web Scraping – Data as a Service.

Have you heard of data as a service (DaaS) before? A concept which can create product and services around big data which eventually can generate big revenue.  I give it a thought a long time ago but recently I started to think more about it; what If it could be done using web-scrapping techniques. In short, giving a service for your customer to generate data based on scrapping tools and technologies.

Web Scraping or Web Harvesting is a technique to extract data mainly from websites. You usually view the websites in the web browser but what If that data is parsed and saved usually on your high end databases and then analysed for deeper, summarised information. Usually you will be using the big data tools to analyse but our main topic here is not about analysing that data but how you can use a web scraper as a service for your clients and be able to give them an access to that scraped data.

Therefore, I started to create my own venture named Scrapify.me. It will cater our user’s need of getting out data from third party websites using our scraping tools. Scraping can help them in many different ways; They can create their own application/startup with pre-filled scraped data for their initial launch, It can help them in understanding the price/item comparison and monitor the changes, It can be used to analyse and study the meaningful data and so on.

Let me know through your feedback about the idea. I am very keen to look forward for your suggestions and comments.

Completion Suggester using Elastica Library

Recently, I was working on the basic example of completion Suggester using Elastica PHP Library. But Elastica doesn’t provide the support for context and completion suggester. The only way right now is to use RAW query with Elastica to achieve the result. I have written down the basic example below. You can add more parameters inside query variable.

 

Let me know, If you find it helpful.

 

Storing user sessions on Amazon Elastic Cache

Here in our company, we moved from Elasticbeanstalk enviroments to our customized servers which are currently behind on manually created elastic load balancer. The reason to move was limited freedom in terms of Scalability. Beanstalk environments were set to auto-scale automatically and It was handling the user sessions through its own auto-created load balancers.

When we moved to our own cluster of servers behind a load balancer, we decided to move the handling of application sessions completely from instance storage to Amazon Elastic Cache service. We are using Memcached engine inside Elastic Cache which is widely adopted memory object cache system and very easy to use.

Now, the next question was how to connect your application with a session handler. We are currently using PHP as our backend language.

Below are the settings which needs to be updated in your PHP.ini

For more info – http://php.net/manual/en/memcached.sessions.php

If you are behind a load balancer, try creating a script which will set this setting up for you at the start of each server.

After this, we have a centralized session storage using memcached.

 

 

Handling Amazon SNS tokens efficiently for Push Notification.

I was working on my company’s mobile app albaloo which is built using Titanium SDK for Android and iPhone. As per our previous architecture whenever our user open our app, whether first time or subsequent times, we were creating Amazon SNS Endpoint based on the token received from GCM and APNS and saving the information into the database along with customer’s geo-coordinates. We were also deleting any previous endpoint created on that token from the database on that same call. We had business requirements to send push notification from our backend based on the geo-location of our customer’s devices.

First approach

To publish, we had a very simple approach first, we were selecting our desired geo-located endpoints saved in our database, subscribing them to SNS topic and publishing a notification to them. Using this approach, we have seen that push notification were not getting delivered fully to all selected endpoints. Below are the couple of issues we encountered.

1) We were not sure whether the selected endpoints are still valid or not.

2) What If the user deleted our app and we still publishes a notification to our stored endpoint ARN.

Best approach so far

So, obviously we had to change our mechanism not only for creating and storing endpoint but also how we can publish them efficiently using Amazon SNS. This approach was mentioned on Amazon Blog.

We need to be ensure about:

1) The Endpoint ARN should exist in our database as well as on SNS.

2) The app/mobile token which is used to generate an Endpoint ARN in SNS must be latest valid token.

3) The Endpoint ARN must be enabled. Amazon SNS marks an endpoint as disabled when a user removes app from his device.

Below is the pseudo code for registering an Endpoint ARN.

I have implemented this pseudo code in PHP over Here

This approach can be used anytime and the registration of the endpoint will never become invalid. Some interesting points mentioned to note about this approach:

1) We are calling SNS API CreatePlatformEndpoint twice. First is when an endpoint is not stored in our database which eventually means its a first time user. Second time, when our stored endpoint ARN doesn’t exist in Amazon SNS.

2) getEndpointAttributes is called to compare the token in Endpoint ARN with our stored token in our database. It also check if the Endpoint ARN is disabled. It will call SetEndpointAttributes to update the latest valid token if the token is not matched and mark the Endpoint ARN as enabled.

I also want to mention that sometimes even after re-installing the app on the device, It will return the same token which was previously used to generate an endpoint ARN which will create a same Endpoint ARN with disabled attribute. we were not handling this case in our first approach.

 

Using this approach to publish, we are determined that It will reduce the # of push notification failures and uses a latest valid token to register even If a person reinstall our app.

If you have any better approach please write down it to me.

Laravel 5 Released, What’s next?

According to Taylor, this release has been the major and ground-breaking release of Laravel. Last year, during the Laracon, Taylor has announced that he will be releasing Laravel 4.3 in around november 2014. But as per his tweet related to it later, he has mentioned that due to some major changes like directory structure, the release will be round up to 5.0.

Some major features of Laravel 5.0

1) New Directory Structure – Only your app logic will go inside the app folder. Everything is outside of app directory.
2) Name-Spacing – setup your app namespace globally.
3) Requests –
4) Method Parameter Injection – This is self-explanatory.

What’s next for me

I would surely give it a shot and test this new version in my separate development branch, but right now, I am more focused on completing my current laravel 4.2 projects. Hopefully, you will see me developing new projects using 5.0 in couple of months.