How to Build Facebook Chatbots With IBM Watson

by Sarang Nagmote



Category - Website Development
More Information & Updates Available at: http://http://insightanalytics.co.in

---

I’ve open sourced a simple sample of a chatbot for Facebook that leverages IBM Watson Dialog and IBM Watson Natural Language Classifier for conversations with users.

Get the code from GitHub.

The GitHub project contains a sample of a Facebook bot built on the Facebook Messenger Platform which is currently available as beta. The sample bot is pretty simple but might give people some ideas about how to use cognitive Watson services and host the bot on Bluemix.

The sample bot shows recent tweets with a positive or negative sentiment about a specific topic via the Insights for Twitter service. Check out the screenshots folder for more information.

The left screenshot shows the usage of the Dialog service. The right column shows additionally the Natural Language Classifier. In this example, users can type in ‘awesome’ which is a word that is not hardcoded anywhere.

I’ve implemented the bot via Node.js. In order to invoke the action that reads the tweets, I’ve extended the dialog definition with the JavaScript code which is supposed to be invoked.

<item>Alright. Here are the {Sentiment} tweets about {Topic}:$ExecCode$showTweets(sender, "{Topic}", "{Sentiment}")</item>

In order to use buttons in the chat, I’ve extended the dialog definition with the button data.

Rather than clicking on buttons users can also type in text like ‘awesome.’ Rather than hardcoding all alternatives, the Watson classifier is used to determine whether users are interested in positive or negative tweets. The id of the classifier is defined as part of the dialog definition.

<item>Are you interested in positive or negative tweets?$ShowButtons$[{"type":"postback","title":"Positive","payload":"positive"},{"type":"postback","title":"Negative","payload":"negative"}]#3a84cfx63-nlc-5285</item>

A Histogram Display Component in Swift for iOS

by Sarang Nagmote



Category - Mobile Apps Development
More Information & Updates Available at: http://http://insightanalytics.co.in

---

While Im writing some new content for Core Image for Swift discussing histograms, Ive found Core Images own histogram display filters slightly wanting. Always eager for a challenge, I thought Id write my own component for displaying histograms and plug it into Filterpedia. The component has no dependencies on Filterpedia, so can easily be reused in any other projects. 

The component is named HistogramDisplay and uses vImage to calculate the histogram data for a given CGImage. The histogram data consists of four arrays (one array for each color channel) of 256 unsigned integers - each containing the number of pixels in the image with that particular value. 

With that data, I use my Hermite smoothed drawing code to create Bezier paths for the three color channels (ignoring alpha) and simply use those paths to draw onto CAShapeLayers. 

The vertical scale is based on the maximum pixel count for any color / tone "bucket". Since there are occasionally outliers that make the scaling a bit crazy, my component allows the user to set the vertical scale with a touch gesture. 

To open the histogram display as a popover in Filterpedia, toggle the UISwitch in the top right corner of the user interface:


Maybe not the most elegant user interface solution, but working for my immediate requirements. 

The source code for HistogramDisplay is available here.

Mission Pattern: Lets Create Fluent Automation APIs

by Sarang Nagmote



Category - Developer
More Information & Updates Available at: http://http://insightanalytics.co.in

---

The Need for Fluent APIs

Automation, automation, automation. It seems to be one of the prevailing mottos of most agile projects I worked for. And rightly so, it makes perfect sense in the era of software and web revolution for us developers to write software that tests software. At the same time, automation brings a lot of value and enables teams to focus on continuous delivery of quality software. 

Having said that, automation is never an easy business. Well, it might be easy (to an extent) to spin up a small automation project to test a few web pages, or unit or integration tests for relatively simple classes; however, its not easy (in the sense that it requires developer traits, patterns and effort) to write maintainable, scalable and sharable automation code. Equally, it requires effort to write Fluent APIs - code that describes business and technical interactions in a human-readable fashion. But the result is worth it!

Developers = Craftsmen

Every agile software engineer should feel and act as a craftsman. As craftsmen, we endeavour to write maintanable and scalable code. And that applies both to production as well as testing code (whether integration, acceptance or unit). This is the main drive to write Fluent APIs.

The Pain...

Page objects or helper classes with hundreds of lines, or unsharabletest artifacts — these are some of the symptoms of automation projects which suffer and quite frequently are abandoned or even written from scratch... 

At the same time, a lot of developers feel the need to de-couple the implementation of their automation from the test execution libraries they use (whether xUnit or BDD-style ones). In that way, they dont restrict themselves to one particular testing tool. 

Thats why we also want to avoid the re-invention of the wheel syndrome: It seems like a déjà vu experience; when you join a new project and you start creating services or tools to enable you perform your automation tasks quite so often, you find yourself copying or re-writing code you wrote in the past. And not only that - parts or whole packages of the classes youve authored, most of the times, can be useful to many more automators like yourself (e.g. connecting to databases and run stored procedures, fetching files from an SFTP location, creating a WebDriver factory, and so on, to name a few...). Admittedly, there are plenty of open source libraries which can help you specify your requirements (Cucumber, Specflow etc. etc.) or run web tests (e.g. Webdriver, Capybara etc.), but you have to write (or re-write) the boilerplate code yourself. Itd be good if we could share Actions and Capabilities, effectively Missions and Tools so as to reduce the need to clone github projects or extend existing classes...

Applying the Mission Pattern can be a means to assist you with all of the above. Lets see how:

The Mission Pattern: Main Concepts

a. The Agent

 The Agent is the main entity that can you can use to perform Business and Technical tasks.

b. The Mission

The Mission is a set of Interactions with the System you want to test. It can optionally return a result. You can use Missions to create technical actions (e.g. ssh authentication to a server, selecting and materialising data from databases) or business actions. 

c. The Tools

Tools are capabilities that the Agent obtains to perform Missions (e.g. a tool could be an instance of ChromeDriver, a Database tool or an instance of a bespoke client library useful to your tests, to name a few).

In More Detail

The Agent is the executor of missions by means of tools that enable him to interact with different layers of your system, or even external systems. In that way, you can perform end-to-end, acceptance and integration tests alike and at the same time create share-able Missions and Tools for your team to use or even other teams in your organisation; and thats feasible exactly because Missions are small, compact classes, focusing on a specific business or technical goal. They, then, fully abide by the Single-Responsibility-Principle. The Agent applies the Visitor Pattern, so he is able to make use of his tools to orchestrate the execution of the missions we assign to him.

How To Apply It

As part of Magentys we have created Cherry-JVM Framework (http://cherryframe.work). It is a micro-libraries framework enabling the application of Mission Pattern. In our documentation section we have a separate chapter with practical examples on how to write meaningful, share-able Missions and Tools in Java. Effectively, its a canvas of tools helping developers to write their own Fluent APIs.

Let the code be fluent. Let there be Fluent APIs.

Use the Correct Data Type

by Sarang Nagmote



Category - Databases
More Information & Updates Available at: http://http://insightanalytics.co.in

---

Blog post #5 in support of Tim Ford’s (b|t) #iwanttohelp, #entrylevel. Read about it here.

Saying that you should use the correct data type seems like something that should be very straight forward. Unfortunately, it’s very easy for things to get confusing. Let’s take a simple example from AdventureWorks. If I run this query:

SELECT a.ModifiedDateFROM Person.Address AS aWHERE a.AddressID = 42;

The output looks like this:

2009-01-20 00:00:00.000

Normal right? You see the year, the month, and the day followed by the time in hours, minutes, and seconds as a decimal. Ah, but there is an issue. This query is supposed to be for the reporting system, and the business only cares about the date that the values in the Person. Address table has been modified, so they don’t want to see the time. Also, the person in charge is kind of picky. They really don’t like seeing the date formatted that way. They prefer to see “January 20, 2009.”

Far too often then, the easy answer, just change the column to varchar. You can trim the time and output in exactly the format needed by the business. Problem solved and it was easy…

Well, until someone inputs “Janry 20, 2009” slightly mangling the spelling and suddenly your report looks all messed up. Or, they ask you to start filtering just the last two weeks, regardless of when the report was run and you now can’t easily do date math on the column. Even after you get over that problem with a little formatting using CAST (along with ISDATE to try to catch all those other typos that are in the system now)  you notice that the performance is really slow so you go to put an index on the column and now you have an index key that is 50 bytes wide instead of the 3 bytes that the DATE data type would have been, making the index less efficient (not to mention, sorting the data is going to put February ahead of January, more formatting).

The list goes on and on. You’re going to hit issue after issue and all you tried to do was a little formatting, make the data look pretty by using the VARCHAR data type in place of DATETIME or DATE. This kind of thing happens all the time, especially to people just getting started. The rules should be very simple, use the correct data type for the data you’re collecting. However, it’s easy to get distracted. It’s easy to get pushed. The business says “put it in this format” and the only way you can do that is by changing the data type so…

Don’t do it. Take the time to understand the implications of mangling the data types. It’s not just date and time that cause issues either. Lots and lots of stuff can be easily stuffed into a VARCHAR field that more properly should be an INT or a DECIMAL or some other data type. Be able to communicate with the business so that they understand that you’re not simply being difficult, you’re trying to protect the functionality and integrity of the data. Show how formatting can be done using client tools. Explain all the downstream issues that this decision will have. Then, use the correct data type for the information being stored.

BigDecimal Gotchas

by Sarang Nagmote



Category - Data Analysis
More Information & Updates Available at: http://http://insightanalytics.co.in

---

Today Id like to share some hints on BigDecimals, how and when to use them, what to take care about, and how to avoid strange, but common errors.

When To Use Them

I think this is the most straightforward of them all. BigDecimals should be used when exact results with arbitrary precision are needed for numerical calculations.

Doubles can cause problems when used for exact answers, as not all floating point numbers can exactly be represented as doubles (see about representation of floating point numbers). Heres the worlds simplest source code to show this:

System.out.println(5.8 + 5.6);

The above code snippet will display 11.399999999999999 instead of the expected 11.4. If this had been some application for monetary calculations, we would have just lost (or won, it depends) a tiny amount of money. 

Ok, so lets fix this. Since this is a post about BigDecimals, our first attempt is to wrap all those double values, like this:

BigDecimal first = new BigDecimal(5.8);BigDecimal second = new BigDecimal(5.6);System.out.println(first.add(second));

And the result is: 11.39999999999999946709294817992486059665679931640625. Again, not 11.4 as we have expected. We are facing the very same issue as before, only with some more digits. This should be no surprise, to quote the Javadoc for the constructor BigDecimal(double):

"Translates a double into a BigDecimal which is the exact decimal representation of the doubles binary floating-point value."

Yep, this constructor really just wraps the double values; it applies no extra smarts to save us from trouble. Lets try it differently, with our numbers expressed as strings:

BigDecimal first = new BigDecimal("5.8");BigDecimal second = new BigDecimal("5.6");System.out.println(first.add(second));

And yes, the result finally is 11.4, the value we were looking for. I guess this is the first morale of this story: one should use the BigDecimal constructor that takes a String rather than a double to translate a floating point value into BigDecimal objects.

When Not To Use Them

This one is pretty simple too. Since BigDecimals are for exact values, they should not be used when an approximation suffices. Thats simply because they are pretty heavyweight.

Lets suppose we want to create a simple application that iteratively approximates the value of Pi. A really old (and I guess outdated by now) formula for approximating Pi (taken from Wikipedia) is:

Now lets try to apply this formula for k=2000, using first BigDecimals and then doubles to see what happens.

First, the BigDecimal version:

private String approximate() { BigDecimal sqrtTwelve = new BigDecimal("3.4641016151377544"); // pretty good approximation BigDecimal pi = BigDecimal.ZERO.setScale(16); for (int i=0;i<LENGTH;i++) { BigDecimal numerator = MINUS_ONE_THIRD_BD.pow(i); BigDecimal element = numerator.divide(new BigDecimal(2*i+1), RoundingMode.HALF_EVEN); pi = pi.add(element); } pi = pi.multiply(sqrtTwelve).setScale(16, RoundingMode.HALF_EVEN); return pi.toString();}

and now with primitive double values:

private String approximate() { double sqrtTwelve = Math.sqrt(12.0); double pi = 0.0; for (int i=0;i<LENGTH;i++) { double element = Math.pow(MINUS_ONE_THIRD_DOUBLE, i) / (2.0*i+1.0); pi = pi + element; } pi = pi * sqrtTwelve; return pi + "";}

I did the usual milliseconds based elapsed time measurement to see how these code snippets perform. I also set my heap large enough to avoid garbage collections during those 2000 iterations, which might be cheating, as it only applies to the BigDecimal version and can be a real overhead in large systems.

Here are the results:

Version	Result	Running time (ms)
Pis real digits	3.1415926535897932	N/A
Big Decimal	3.1415926535897931	8315
double	3.141592653589794	3

Well, both versions approximated Pi really well. The BigDecimal version got 15 decimals right, while the double version got 14. However, the latter outperformed the first by more than 2771 times. In this case, the result were getting using doubles is more than good enough, and should be considered instead of heavyweight BigDecimals, given the huge performance gain.

What to Remember

Lets create a simple JUnit test file to observe the behavior of BigDecimals compared to Doubles. It looks like this:

String pi1 = "3.14";String pi2 = "3.140";String pi3 = "3.1400";@Testpublic void test_doubles() { Set<Double> set = new HashSet<>(); set.add(Double.parseDouble(pi1)); set.add(Double.parseDouble(pi2)); set.add(Double.parseDouble(pi3)); assertEquals(1, set.size());}@Testpublic void test_bigDecimals() { Set<BigDecimal> set = new HashSet<>(); BigDecimal bigDecimal1 = new BigDecimal(pi1); BigDecimal bigDecimal2 = new BigDecimal(pi2); BigDecimal bigDecimal3 = new BigDecimal(pi3); set.add(bigDecimal1); set.add(bigDecimal2); set.add(bigDecimal3); assertEquals(1, set.size());}

If we run the test cases above, we can see that the first one passes with no problem, while the second one fails with a message: "expected 1, but was 3". Even though 3.14 is equal to 3.140 and 3.1400, we are still getting strange results.

Lets see why. When we created our three BigDecimal objects, we specified different number of decimals (2, 3 and 4, respectively). When passed to the constructor, that number of decimals got translated to a desired level of precision, also known as scale (observe calls to setScale(...) in the previous example too). The first instance would have a scale of two, the second three while the third four. BigDecimals equals() method takes into account both the represented numeric data and the scale. Even though equal from a mathematical point of view, our instances differ in scale, so the equality check fails. Thats why we end up with three elements in the set instead of one.

If, however, we set equal scales for our three BigDecimal instances like this:

@Testpublic void test_bigDecimals() { Set<BigDecimal> set = new HashSet<>(); BigDecimal bigDecimal1 = new BigDecimal(pi1).setScale(4); BigDecimal bigDecimal2 = new BigDecimal(pi2).setScale(4); BigDecimal bigDecimal3 = new BigDecimal(pi3).setScale(4); // this is redundant, btw set.add(bigDecimal1); set.add(bigDecimal2); set.add(bigDecimal3); assertEquals(1, set.size());}

Then everything works fine again; both the numeric values and scales are equal so the instances are considered equal.

To make this even more interesting, lets remove the setScale(4) statements and change the object type of our set from HashSet to TreeSet, like this:

@Testpublic void test_bigDecimals() { Set<BigDecimal> set = new TreeSet<>(); BigDecimal bigDecimal1 = new BigDecimal(pi1); BigDecimal bigDecimal2 = new BigDecimal(pi2); BigDecimal bigDecimal3 = new BigDecimal(pi3); set.add(bigDecimal1); set.add(bigDecimal2); set.add(bigDecimal3); assertEquals(1, set.size());}

Surprise, now the test passes again. Thats because unlike HashSet, TreeSet is a SortedSet, where element equality is based on the outcome of the compareTo(), rather than the equals() method. BigDecimal is one of those classes in the JDK where equals() and compareTo() are out of synch. compareTo() does not require the scales to be equal for considering the instances equal, hence we end up with one element in the set (just as we want).

In order to receive mathematically correct results, BigDecimals should always be compared using the compareTo() method.

Oh, and one last thing, just for the sake of completeness: BigDecimals are immutable; every modification results in a new object being created. That is why the following code is broken, and prints "Whoooops" instead of "Math makes sense!":

BigDecimal bigDecimal1 = new BigDecimal(0);BigDecimal bigDecimal2 = new BigDecimal(10);for (int i=0;i<10;i++) { bigDecimal1.add(BigDecimal.ONE); // this should be bigDecimal1 = bigDecimal1.add(BigDecimal.ONE);}if (bigDecimal1.compareTo(bigDecimal2) == 0) { System.out.println("Math makes sense!");} else { System.out.println("Whooops");}

Four Reasons Why Python Is a Good Programming Language

by Sarang Nagmote



Category - Website Development
More Information & Updates Available at: http://http://insightanalytics.co.in

---

When I started programming, Java was all the rage. There were no doubts regarding this. Java is the only programming language in the information technology industry that has remained strong since it’s inception date. Java is the most successful technology in the industry and this is due to its versatility. Java can be used to develop anything possible from software applications, desktop applications, mobile applications, gaming clients, and web applications. It does this all without too much hassle, also.

However, this is all changing, and its changing fast. Python is starting to take the lead for all of Java’s greatness. Python has managed to remain in the shadows for more than a decade since it’s own inception. Yet now, it has emerged as an invaluable gem for the technology industry. It is receiving amazing popularity and acceptance due to its performance, efficiency, and secure environment.

All of these aspects have compelled the industry to use Python, with many programmers having relied on this technology for the past couple of years. Besides these basic features, I listed in the second paragraph, there are numerous other abilities that are incorporated within Python. This makes it superior for a million different uses and helped it gain trust from developers worldwide.

Python is Easy to Learn

For a new programming language to take hold and be accepted by the ever-changing technology industry, it has to be simple to learn and easy to apply. If a language is difficult to learn and integrate, it will have little chance of being widely adopted. Heck, Python even uses real-time wordings, making it easier than most to pick up.

Almost all of the syntax is very similar to "real-life" words. The keywords are very familiar to any non-programmer, so beginners can pick up on the language without too much difficulty.

Python is Super Efficient

Efficiency means everything to a new programming language past 2010. A new technology lacking efficiency will never gain the same success as Java or Python. We have to accept the standards set by Python because of the libraries and methods that were created for us to use in our source codes. With these built-in methods and classes, common tasks are easily executed systematically without hampering the application.

Python is Highly Scalable

In the last four or six years, the industry has seen the requirement for highly scalable methodologies (such as Agile methodology). When a client requires a large amount of development for a software application that does not require for the project to be complete, the application can be moved to live at any moment of the development process by completing a certain amount of modules of the application.

The application remains easily open to being connected to other new modules in a systematic approach without disturbing its existing modules. This enables the application to remain in an auto-complete mode so that it can be enhanced further in later periods of time without debugging its core too much.

Python is Always Updated

A new technology often seems to depreciate in less time than it takes to be developed. Everything comes with an expiration date, always being replaced with a better (or sometimes not) solution in the future. If the technology manages to span a number of years keeping consistent or growing in popularity, this implies the acceptability of the technology. Due to its repeated adoption and commitment to refining itself over time, Python definitely seems to be succesful.

Not only that, it has been accompanied with HTML5, CSS5, jQuery, AJAX, and other popular and futuristic technologies. This allows for it to be an amazing choice for highly efficient web applications, smartphone apps, and native applications for any operating system that you can think of.

Final Thoughts About Python

Finally, any technology with high reliability will gain respect and confidence of developers. Luckily, Python passes with flying colors as a highly reliable technology. Python’s reliability lies in its secure environment, availability of reliable resources, and availability of strong support community in the industry.

And, before I close this piece, I have to applaud the Python community. They give on-demand support for their developers anytime—24/7. So, without a doubt, the support and reliability of this technology are superb.

4 Things to Keep in Mind Before Choosing a Payment Gateway

by Sarang Nagmote



Category - Mobile Apps Development
More Information & Updates Available at: http://http://insightanalytics.co.in

---

With the world going online now there is no dearth of online payment gateways. People have readily accepted the online medium to carry out their shopping and banking activities. It has shown some significant growth in the online monetary transactions.

Back in 2011, PayPal, one of the leading global payment provider revealed that they carry out $315.3 million in payments per day. Talking about the holistic business transaction back in 2013, more than $1200 billion of eCommerce transactions.

The entire credit of this success goes to the reliable working methodology of a payment gateway along with the web and mobile app developers that comply with PCI Data Security Standards. It ensures that payment procedures are performing safely and speedily. Therefore, it is necessary to invest some analytical thoughts when it comes to choosing the integrated payment gateway.

Not only this but hiring the best app development companies can solve the problem by integrating payment gateways even with the mobile apps. The reason being, integrating correct payment gateway makes it easy for customers to pay for the products or services via the web or mobile app. Here are a few things vendors must bear in mind while choosing the payment gateway.

1. Security

The topmost priority while choosing the payment gateway should be the payment security. With the advent of online transactions, there is also a significant increase in numbers of credit card frauds and dummy transactions. The Payment Card Industry Data Security ensures that customers’ virtual money data is collected and stored securely, thereby reducing the security risks associated with the online trades.

It is always better not to assume that the payment gateway is a PCI compliant. Because complying with these standards are not only merchant’s responsibility as your business is also involved with the same. It is always advisable to ensure that your security requirements are possible to meet, and best is to inquire about the compliance surrounding them.

One of the most important aspect to consider is the method of capturing credit card data which involves choosing from the iframe, hosted, or API. Also, it is best to know about the information safe-keeping protocols followed by the selected methodologies.

2. Choosing the Type of Gateway: Classic V/S Modern

The basic difference between the Classic and Modern gateway is merchant account. Classic gateway requires the merchant account while Modern gateway does not need the same as the same is seamlessly in sync with the bank account. To ensure that payment gateway remains in harmony with the web and mobile app one need to hire top app developers who can understand your requirements precisely.

Classic Payment Gateway

Opt for the Classic Gateway if you are ready to tackle some hassles and run a bigger store. Classic gateway is best for the business which operates on the bigger scale with the high amount of daily transactions. However, there are plenty of hassles in deploying the Classic Gateway.

Apart from taking much time to set up, it also requires some amount of technical knowledge because the same needs to be integrated with API and requires a merchant account. One of the biggest advantages of the Classic Gateway is the lesser amount of per-transactions fee compared to the modern ones.A few popular examples of the Classic Gateway are Authorize.Net, WorldPay, and 2Checkout.

Modern Payment Gateway

One of the biggest advantages of Modern Payment Gateway is that you do not require a merchant account, and it is comparatively easy to set up. However, at the same time, the per-transaction rates are relatively more. One more disadvantage is that it redirects customers to the payment site which can hamper the conversion ratios. A few of the popular examples of modern gateway includes PayPal, Stripe, and SimplifyCommerce.

3. Transparency

The online payment industry has witnessed some significant change in the trading since past 5 years. Nevertheless, historically speaking, the industry is full of companies hiding the fees and locking you on long term basis.

Off late, there is one thing which everyone in this business is looking for - transparency. Transparency is now the primary concern for the businesses dealing with the virtual money and online transactions.

The best way to gauge the service provider is by it is clear and simple pricing plan. It is of utmost importance that the transparency is maintained throughout the business deal, allowing you to leave or opt out for a different provider at any point of time by letting you take the credit card data with you. There are few of the vendor who gives the services of data portability.

4. Selecting the Best-suited Hosting Methodology

Regarding hosting the payment gateways, there are two options. One is the off-site or a hosted method, and the other is integrated gateway.

Hosted Payment Gateways

People familiar with the PayPal must already be aware of the hosted payment gateway that redirects the customers to the payment processor’s platform where users give out all the payment information. One of the best rewards of the hosted payment gateway comes in the form of leveraging responsibility as the offsite provider is solely responsible for the PCI compliance and data security.

On the downside, in some countries these offsite payment processors are not trusted while on the other hand in some they are more preferred. So the best way is to gauge the user’s perception of how they perceive a hosted gateway. Moreover, also, diverting the users to another website is not a good idea, especially if they think that the particular gateway is not trusted.

Integrated Payment Gateway

Integrated Gateway connects the eCommerce site with the gateway’s pre-delivered API. The disadvantage of customer going off-site in hosted gateway turns the same into an advantage as they never go off-site for the payment purpose. In all, it gives a smooth and seamless experience to customers.

Even the integrated gateway has its limitation in the form of the ability required to integrate the chosen gateway. There are plenty of eCommerce platforms that can smoothly incorporate a wide range of gateways with the help of various APIs. However, there is one protocol over here, choosing the ones which do not support “out of the box” may require some custom programming for its efficient functioning.

Moreover, the pricing factor should also be considered while selecting the payment gateway. It is important to gauge the profit from sales as it can affect the businesses dealing in luxury goods. It is always better to hover upon the requirement and capacity multiple times before deciding the payment gateway for the business offering websites.