#web-service

This post is to share my experience in building complex Apps over third-party web services. The post focuses on design challenges when building batch-processing services,

  We all know that when measuring performance there are two things to consider.

One is the cost of operation and the other the frequency of operations. 

  for ( Object o : objLst ){ // freq of operation

  o.doSomething();      

  Response resp = makeWebServiceCall(o);  // cost of operation

  processResponse(resp,o)

}

  Where the cost of operation is usually not considered by a programmer.

We focus only on reducing the frequency of operation using Algorithms and Data structures.  This works fine cause usually the cost of operation usually depends on the underlying hardware and can only be improved with better hardware. But when it comes to building Web services, which call other third-party services in synchronous fashion the cost of operation is very high, as latency cost could grow based on the location of the third-party service server boxes.

  Hence comes the need for batch calls.  The role of batch calls is when making POST/PUT call to the REST service to create/update/delete.  The idea is to avoid making N independent calls for N independent entities and batch them up.

  This involves creating services that build Array of entities and send all these to the third-party service in one shot.  Now this requires a little different approach to the problem.

  This involves more iterations to preprocess the data make single web service call and post process the data and also requires proper error handling at every stage.

Hence there would more loops to do each of these preprocessing steps but this would still be very efficient then making N web-service calls.

Error Handling becomes tricky when operations are transactional. Consider the case when:

 Your single action needs to

1)   Create List Entity A objects.

2)   For each Entity A create List of  Entity B and One Entity C

It's no secret that I am a huge fan of Web APIs. Working with Web APIs is full of fun. Below is some of APIs for you:

VNTextPro

NPL API for Vietnamese: word segmentation, sentence segmentation, post tag. Support 2 engines: vnTagger & JVnTextPro.

Hosted by Heroku

http://vntextpro.api.olragon.com

OpenOCR

OpenOCR is open source OCR-as-a-Service using Tesseract and Docker.

http://openocr.api.olragon.com

PortScanner

PortScanner is PortScanner-as-a-Service written in node.js. Blog post (Vietnamese) http://vietjs.com/node-js/quet-cong-mang-sieu-nhanh-su-dung-node-js/

A nice article on web services