Mingfai's blog

Startup by definition is high growth and scalable. Most "startup" has to be technology-driven, I believe.

Startup people tends to believe in an early stage, rapid development and time to market are critical, technology is less important and its ok just to use whatever that enable rapid development. It is a common lean startup concept to build fast and release to the customer to get feedback. Of course I wouldn't disagree. 

However, technology choice affect development time and cost, and enable different product feature. To achieve a sustainable business model (e.g. create enough value to raise capital/loan, or break-even/revenue>cost), technology cannot be ignored. One may prove a market and validate an idea without technology consideration, I don't believe one can create a successful technology-driven startup without being good at technology.

Assume startup is a multiple step (milestone) process as follows:

(notice that this post is not to develop any model or suggest there are N steps. the steps are only for illustrate a point that will be mentioned at the bottom)

#1 idea, planning, forming a team, MVP/prototype development, customer validation etc.

#2 release to public, customer feedback etc.

#3 on-going refine the product base on feedback, growth

#4 break-even

#5 make a decent amount of revenue

...

#10 create the first 1B value

Lean startup methodologies really help in the first few steps. It is especially important to those who never do startup to take the first step. 

If we do fast, good at prototype and rapid development, it can really help in speed up the learning cycle in the first 2 steps. But it doesn't necessarily help in step #3-5. 

My point is, if one just want to demonstrate he is good at startup such as aim at being a startup trainer, it's fine just to use whatever technology to build something. 

if one want to get to step #5, one need not only a validated idea/business model but also do the product properly with technology.

Edited:

- of course, "doing technology right" obviously do not mean invest a tremendous amount of time and money in technology, or spend 3 months in building infrastructure without delivering anything concrete... if it is not clear enough

- "doing technology right" does mean a business founder shall invest a lot of time to in hiring the tech team or picking a right tech co-founder. 

- "doing technology right" to the tech founder, is to spend time picking the right set of technologies suitable for use in a startup environment. 

A simple case as an example:

Rule 1: if a cashier forgets to enter a price, set to $10

public static class SetToTenDollarsIfZero    implements Function<Double, Double> {   @Override public Double apply(Double in) {     double out = in == 0 ? 10D : in;     log.trace("{} -> {}", in, out);     return out;   } }

Rule 2: Apply a 20% off anyway

public static class TwentyPercentOff    implements Function<Double, Double> { @Override public Double apply(Double in) {

    double out = in * 0.8;     log.trace("{} -> {}", in, out);     return out;   } }

Build the rules

Function<Double, Double> rules = Arrays.asList(   new SetToTenDollarsIfZero(),   new TwentyPercentOff() ).stream().reduce((a, b) -> a.andThen(b)).get();

Run it:

double result = Stream.of(0D).map(rules).findFirst().get(); log.info("result: {}", result);

Output:

SetToTenDollarsIfZero | 0.0 -> 10.0 TwentyPercentOff | 10.0 -> 8.0 result: 8.0

Remarks The following lambda expression for reduce(): "(a, b) -> a.andThen(b)" is self-explained. andThen() is a default method of the Function interface. (default method is new in Java 8, btw) reduce() is to reduce a stream with a reducer function. (well...) (a,b) - take two parameters as input, return one. 

Improvement map() is for one-to-one mapping. In many cases, flatMap is more useful as that return 0, or 1 or many results. Say we want to add the 3rd rule to drop any order below $0.

Change the original rules to "implements Function<Double, Stream<Double>>", e.g.

public static class SetToTenDollarsIfZero   implements Function<Double, Stream<Double>> {   @Override public Stream<Double> apply(Double in) {

    double out = in == 0 ? 10D : in;     log.trace("{} -> {}", in, out);     return Stream.of(out);   } }

Add a third rule:

public static class DiscardIfUnder5Dollars   implements Function<Double, Stream<Double>> {   @Override public Stream<Double> apply(Double in) {

    return in == null || in < 5 ? Stream.empty() : Stream.of(in);   } }

Build the rules with a slightly different syntax:

Function<Double, Stream<Double>> rules = Arrays.asList(   new SetToTenDollarsIfZero(),   new TwentyPercentOff(),   new DiscardIfUnder5Dollars() ).stream().reduce((a, b) -> a.andThen(d -> d.flatMap(b))).get();

The way to call it is slightly different as well: double result = Stream.of(0D).flatMap(rules).findFirst().get();

Remarks: filter() can do dropping as same as returning an empty, but is no good for this case. if the requirement is just to send an alert rather than do dropping, peek() may be used. peek() works like a listener that process on the course of the stream, but do not alter the return value.

If I'll write a part 2, it will cover:

Define the Rule interface with a weight/priority default method

Use service loader or other mechanism (such as Guice Multibindings) to load a list of Rules. 

Two Java 8 usages:

Turn Iterable to Stream: Iterable<Callable> calls =ServiceLoader.load(Callable.class); //false -> no parallel  Stream<Callable> stream = StreamSupport.stream(calls.spliterator(), false);

Hide checked exception in stream: stream.flatMap(Wrapper::call).forEach(System.out::println);

static class Wrapper {   public static <T> Stream<T> call(Callable<T> callable) {     try {       return Stream.of(callable.call());     } catch (Exception e) {       //log.error(e.toString(), e);       return Stream.empty();     }   } }

of course I use it as a one-liner. :-)

Edited: The choice of Callable and ServiceLoader as example is to illustrate the use of a pluggable design. If you do use Callable (I don't, define an interface properly instead), you can use :

Migrated server project to Java 8 for a few days. Java 8 gives a few benefits so far:

Lambda expression simplifies the code

Time API replaced Joda Time

App start up time seems to be faster, but it's not a big deal anyway except a slight improvement in development. 

The new Lambda stuff is powerful, but still not as rich as Groovy. e.g. Groovy supports eachWithIndex, with etc. That allows me write my code in a way precisely as I like. But the predictable performance makes me choose Java over Groovy.

Take an examples how Lambda simplifies my code:

boolean handled = false; for (Handler h : handlers){   if (Boolean.TRUE.equals(h.handle())){

    handled = true;     break;   } } now it becomes: boolean handled = handlers.stream().anyMatch(h -> Boolean.TRUE.equals(h.handle()) );

There are countless examples about how Lambda can make code cleaner. It does come with some (possible) drawbacks:

One liner is harder to troubleshoot when it throws exception. Stacktrace seems to be able to pin point which Lambda got exception:

Java8Test.lambda$2(Java8Test.java:25)

$2 refers to the 2nd one.

Steps for Chrome users to remove FB ADs

Install TamperMonkey Chrome extension

Copy the "FaceBook Ads Bye Bye" script and paste to a new TamperMonkey script

Why remove?

I love those ADs and happy to see FB makes money. It's a pleasure to see those pretty girl photos (provided by friend matching site) or super hot anime models (from online game companies), and keep abreast of the latest Samsung products. But I don't like they put it as sponsored/recommend-by-friend as its noisy when I want to quick scan all social posts by "friends".

Two ways to remove

FB website, technically, is just HTML. HTML DOM elements can be manipulated during/after it is loading/loaded. There are many browser plugins that can do the job.

Browse plugin/extension  It's more convenient to use a browser plugin/extension. If you pick the right plugin/extension, the script is the same anyway. General users can just pick a popular and high rating AD block plugin.

UserScripts.org For me, I go for UserScripts. For Chrome, I use the TamperMonkey extension. Using user scripts is slightly more complicated. Why do so?

Number of maintainers: userscripts can be used by multiple browser platforms. So i believe there are the best scripts are maintained by more people. 

As I know JavaScript, during the script "installation" process, I can review the script and make sure it is not harmful.

Just wrote an "essay" about number of Android apps vs RAM usage in a FB comment. http://goo.gl/A9BJt

In short, IMHO, it doesn't necessary help. My key points:

officially, you should rely on Android platform to manage memory for you

Uninstall apps free storage space. Lack of storage space can cause your device slow.

Change memory threshold is a more effective way to affect Android's memory management 

Remarks: I'm an Android developer using 3 different physical devices. One of it installed 300+ apps. Another one is old and need to use Cyanogenmod rom to upkeep its performance. I monitor memory usage of these devices seriously with different apps.

My "Essay"

>很多人使用Apps後，只按返回鍵登出，實際上未完全關閉程式，程式依舊在後台運行並佔用手機記憶體（RAM），拖慢手機運行速度及增加耗電量。

Android is designed for people to just press home or jump to another app. The Android platform takes care of memory management (do garbage collection) and killing running apps when there is not enough memory. (free memory fall below a threshold) Google wants user to be able to switch between apps quickly, it is a fundamental design difference between Android and (at least older version of) iOS.

For Android, "exit" an app means you terminate it (kill its process). However, Android has a concept called BackgroundService, that many (if not most) app use. BackgroundService is allowed to restart after you manually kill it. And if you kill it for more than once, the platform will lower its priority (sth like that) and then it can't restart automatically.  If an app has no background service that it won't restart, or even if its background service restart, in both case, your termination will free up some memory. Manually terminate app is not an officially recommended solution AFAIK, not because it can't free up memory, but because you don't need to manually do so as the platform will take care of it. 

Uninstall app is good to reclaim storage space. When storage space get to a low level, it will make a phone slower. But its impact to memory is quite complicated and IMHO, you actually can't get more memory by uninstalling apps, for most cases. When you install an app with background service, the app may use some memory in the background if the platform allows it. Because modern phone has sufficient memory, it usually allow some apps to run in background (so that when you switch to/start it, it starts faster and give a better UX). Let's say the phone has 1,000Mb memory, the free threshold is 200MB, it always allocate 800Mb to allow apps to run in the background. Let's say each background app uses 40MB, so it allows 20 background app anyway. So if your have installed understand 20 app, your uninstallation may affect memory usage. And if you have 40 apps, you need to uninstall 21 apps for your goal to influence memory management to take effect. (and I somehow thing it still won't help because the platform will just use the memory for other thing)

Anyway, the point is, the Android platform claims to manage memory for you, and it wouldn't let unlimited app to run in the background that affect you. If the RAM is slow, or one needs to run an app that has to use a large heap on start, a better recommendation is to adjust some of the memory threshold settings.

In short, IMHO, the following statement is over simplified for Android:

很多人使用Apps後，只按返回鍵登出，實際上未完全關閉程式，程式依舊在後台運行並佔用手機記憶體（RAM）

Hello Ming Fai,  Congratulations! You have passed the AWS Certified Solutions Architect - Associate Level exam.

Just added the 24th certification in my portfolio, since 1999. Some certifications require more than one examination, e.g. "Red Hat Certificated Engineer" and "Sun Certified Enterprise Architect", both took in 2002, are 3 parts/exams. 

It sounds like a lot but I actually didn't take any cert after the 23rd one that is "Oracle Certified Associate (OCA) on Oracle 10g Database" in 2005/11. 

So how and why I take the Amazon Web Service (AWS) certificate exam? 

Late last year, I entered into a recruitment process for a Solution Architect position of AWS SG. Before that, I was experienced with some of AWS's key services since 2009, but have no idea about most of the other products.  The recruitment process covered 2 written/hand-on tests that I need to deliver a solution design, and then I was forced to read/watch a lot of product info, documentation, and video in a short time. 

3 days ago on 2013/06/03, I got the AWS newsletter that announced the certification. Checked the exam content and it seems easy to me with my "preparation" in the last year, and it costs US$150/HK$1170 that is affordable. Then I made a booking on the earliest day, i.e. 2013/06/06 (today) 

Cloud computing is cool! AWS is the market leader.  Nowadays, I dare not to tell I got some relational database certification (Oracle, DB2, and MS SQL) as NoSQL is more trendy. Similarly  Microsoft/Cisco's certifications are so yesterday.

An "architect" exam generally ask about concepts that are relatively easier than implementation. (like CCNA, Java etc.) There were 55 questions, that I marked 32 as not 100% sure for review at the end. Indeed, I'm not that confident to pass and spent like half a hour to try to memorize unsure items to prepare for a 2nd try. :-P

The exam is about architecting solution with AWS's products and it covers many different products. You don't really need to be an expert of every product, but actual AWS usage and general architect experience will help a lot. I wouldn't say it is hard, but there should be no way one can pass without knowing a lot of stuff.

Played with Kotlin for a 1-2 weeks, first tried to use it for building Android app, and then tried to use it to implement Java POJO only as a lib. At the end, I decided to remove all Kotlin from the program and use Java only.

I am looking a replacement of Java language in Android project, and want to see if Kotlin can do the job. At this time, Scala is probably the best alternative in maturity and adoption, but it is a bit too complicated (for its runtime and syntax) and I believe Kotlin is a simple version of Scala with better IDE support. 

My experience of Kotlin as a language is very good. It works with Java seamlessly as same as Groovy. It's more concise that Java, their APIs and language usage are similar that makes it very easy to use. And I like expressive language style. Code readability/maintainability is usually more important that performance and rapid development to me, so I love the Kotlin design philosophy very much.

However, Kotlin on Android is near unusable to me. The trend of Android frameworks, in contrary to Java framework, is to implement compile time operation with annotation processing rather process in runtime, and Kotlin will not support Java annotation processing (in foreseeable future). A number of Android frameworks/libraries won't work (and I can't live without frameworks!):

Otto and AndroidAnnotation that uses annotation processing won't work

Dagger uses annotation processing as well, and even if it doesn't, Dagger field injection doesn't work due to the default private accessibility Kotlin POJO can be used with dagger, but it is a must to use constructor injection. I don't like frameworks or tools that force a specific style of coding.

Otto subscribe won't work, either the same compile time mechanism, or Kotlin doesn't generate a method with "public void" signature.

I believe I'm a lucky one who is happy with Java as programming language, with good experience of it, and also able to write concise and maintainable code. So I just check other language to see if they are better than Java language, but I'm not desperately to look for a replacement.

The first post shall be an overview. I'll focus on Java and web software development.

What is quality?

Quality is not scope or functional requirement, it's not project schedule (it does affect schedule, however), usually not something typical users know how to ask for.

I learned 8 quality factors from the very old Sun Certified Enterprise Architect [1] material, roughly as follows:

Usability - a.k.a. User experience in modern term

Reliability

Scalability

Extensibility

Performance

Availability

Security

Maintainability

From the wikipedia Software quality page, there are also testability that is quite important. Some of them can be specified as a technical/non-functional requirement, and related to each others. Many of them are about architecture design. Most of them are costly to measure.

Let's look at them from the users' point of view. Usability directly affect the users. Reliability(is it buggy?) and performance (is it fast?) and availability (is it down?) are noticeable by users. Extensibility affects users when they make change request or have new requirement. The remains security, scalability and maintainability (and also testability) do not usually have direct impact.

How to measure quality?

Let's look at them one by one:

Usability Quite hard to quantify. One approach is to do A/B test and see which group has a better metrics like click-through/retention rate. 

Reliability Count the bugs in issue tracker. High number of bug report after a production release probably reflect a poor testing process. A bug discovered in a later stage is more costly to fix. 

Performance It is much easier to measure production performance nowadays. In the past, it's easy to measure server resource like porting a chart for CPU, RAM, Disk and network usage, but that do not necessary reflect application performance. Nowadays, framework like Yammer Metrics make it very easy to collect metrics, and solution like Graphite make it easy to visualize real time data for monitoring. Before production, in theory, a team can do load test to measure performance. I hope I'll have time to write how I did a very serious load test 10 years ago, and how I don't have time to do a meaningful load test in the next 10 year.  Micro-benchmark may be more practical for performance measurement, but there are a lot of limitation. 

Availability It is perhaps the easiest one to measure. Every component can be monitored. In addition to resource usage monitoring, threshold alert, and reporting, abnormal event can be detected to reduce chance of outage. Modern public web service usually publish the availability history online. Transparency is the way to build confidence in customers.

The other quality factors are harder to be noticed by users. And more able development process and quality in design. They will be addressed in the next post.

[1] - Oracle bought Sun, and the certificate is renamed to "Oracle Certified Master, Java EE Enterprise Architect". 

It's a bad name because "Enterprise Architect" generally refer to those who use enterprise architecture framework to design IT strategy, but not specific to Java development

"enterprise" or vendor-promoted technologies are generally not as good as agile, open source, industrial adopted solution

Related to the previous post, when a project is got into trouble,

the project deadline usually cannot be postponed

adding more resource is not an option  e.g. hiring is time consuming, tasks may not be run in parallel and tasks have leak time etc.

Yes, you can't cut quality! Because quality is not in the formula from the beginning or have been sacrificed already :-)

It sounds unprofessional to say so, but the most powerful technique I've learned in PM is to negotiate with users to reduce scope. And it's not hard because most stack-holders would want the project be labeled as "successful" or "deliver on time".

(I'm not intended to talk about PM but can't stop right here to leave an impression that I planned projects badly and have to do rescue all the time. :-) )

so how to avoid the situation of project overrun? Let me share 3 subjective views focus on the "time" factor, that do not often mentioned in text book:

Do in agile, release in short iteration task break down is the key to project estimation. long waterfall project is hard to break down, and at the end you can just put some task items with man-day figures by guilt feeling, that will just go wrong.  A software project manager without solid tech background may be a great manager but would not be able to do estimation well. 

Do risky tasks first as proof of concept (PoC) For tasks that is beyond the skill-set of the current team and require R&D works, they are high in risk and should be planned as POC tasks rather than just assume a duration (even without buffer). Before the POC is done, there is no way to have a reasonable estimation. Estimation itself is a task that takes resource and time.

A project management book i read many years ago says there are 4 control factors:

time

budget

scope

quality

But, quality is often the neglected factor. I don't often dare to talk about quality to my boss or business people or partner but just do it secretly.

Just a few days ago, I watched a youtube video that uses the term "Engineering Triangle", that officially removed the quality factor from the formula. :-)

Remarks: quality is still in the Project Management Book of Knowledge as a 6-factor triple constraint star. 

Just resurrected my blog at a time when most bloggers no longer blog but tweet. At a time when posterous.com (a large blog service provider) is pulling the plug.

"Posterous Spaces is no longer available" (And tumblr doesn't allow me to  make this line as image alt text)

Every coin has two sides, the cons of Groovy and Grails are:

Performance Performance is its number one drawback. The next 1.6 release will make some improvement, but it's still expected to be much worse than Java. It shouldn't be difficult to make your application 1000 times slower than Java if Groovy applications are not developed carefully. There is a recent thread in Groovy user list that talks about it: http://www.nabble.com/Groovy-performance-...-ouch%21-td16528369.html There is a programming platform implementation benchmark site and it shows Groovy is significantly slower than Java and PHP, and slower than it's "competitor" Ruby and JRuby. http://shootout.alioth.debian.org/gp4sandbox/benchmark.php?test=all&lang=groovy&lang2=ruby It's understandable to be slower than Java which is one of the highest performance language. (it's traditionally known as bad in performance but the situation has changed for a long time due to the significant improvement and maturing of Java Virtual Machine implementation) But it's really sad to know it's slower than the other dynamic/scripting languages. Notice that, for bad developers, they can develop very bad performance Java application; for good developers, they can develop Groovy that doesn't significantly outperformed by an equvilent Java application. So, it translates to monetary cost in human resource.

Development Tools Lack of good development tools is the second major weakness. Eclipse provides basic Groovy and Grails development but it lacks of common features such as refactoring. In terms of development tools, it's again far behind Java. (It's obviously not a far game as Java has been dominating for over 10 years) IntelliJ IDEA is currently the best for Grails development but it's a commercial IDE. This weakness extends to usuability of features such as debugging. IDEA is good enough with some configuration but I don't feel it's as good as debugging in Java. For example, due to its dynamic nature the debug "go into" will take more steps, but it's fixable in IDEA by configuring a filter.

Conceptual drawbacks Some people think dynamic language is evil. The untyped nature makes it hard to develop and maintain as you don't know the actual type until runtime. It's fast and it encourage rapid development but not careful planning and modeling. I would think this is actually a strenght rather than weakness. The beauty of Groovy is it support both static and dynamic features. You can define variables in typed and untyped way. You basically can use it as same as using Java. (which is very stupid as it incurs significant performance cost! :-) )

Get back to an old question, asked in about one year ago, why bother to startup my company?

The strongest force is accountability, contribution, and result to rewards:

Accountability As the boss, you are fully accoutable for the result. You always have no excuse. As a staff, you are responsible for a scope that you are assigned to, it's not your fault if the company is not making profit, and usually you get your monthly pay check no matter how good you perform. Decision making links to accoutability. You can make decision only for the areas that you are responsible to. If you see something in the company isn't go right, there are not too many things you can do. When you run your company, you have influence in most areas. And your exercise your decision making in a lot more area that create higher impact.

Contribution All company welcome employees to contribute, but the employee system doesn't effectively rewards contribution. If you do your best and achieve a good result, you may get a bit more bonus than your colleague who do his job averagely. There is an invisible ceiling that discourage you from significantly outperform other people. At the down side, if you did badly, your company will have tendency to keep you as the cost of replacement of headcount is high, and they can hardly associate any bad result with an individual. Your company bear a large part of the risks and is able to offer you (as well as any staff who may not perform too well) a stable monthly salary anyway.

Rewards Your competence directly affect the result, and the result should directly link to your rewards. Unfortunately, the employee system doesn't work in this way. To be succesful as an employee doesn't necessarily require good skills at work, but the skills to manage your boss. (I don't think shoe-shrining will work, however) Conceptually speaking, a job that require you to specialize on areas out of the nature of the job is a waste of resource. The actual difference in the way of rewarding is that the owner of a company own the shares and any asset. Employees are travellers in a ship. They could stay in the ship for their lifetime but they are not a part of it. It is only when you own the ship, you have natural tendency to make it better.

It's a shy for me as a leader of a Java User Group to promote Groovy which is another programming language derive from Java but is not Java. It's more a shy for Sun Microsystems not able to make Java as simple as Groovy! :-)

So why Groovy and Gails rules?

It provides significant productivity boost. I say 5 to 10 times. It is achieved by its compact and empowered syntax, and it's nature as a dynamic language (e.g. untype declaration)

It seamlessly integrate with Java. Groovy programs are compiled to Java bytecode. Java programs can call any Groovy programs like calling another Java program, and vise versa. So, Groovy can also enjoy the rich set of Java libraries and frameworks in the market.

Groovy has similar but simplier syntax than Java and it's easy for Java developer to learn.

Groovy runs on Java platform which has been developed for over 10 years and is very mature. Groovy web applications are Java EE application that can be run on any application server. At this point it differentiate against its dynamic counterpart Ruby on Rails

Groovy can be used in a wide range of areas: other than general development, it could be used for configuration, dynamic scripting by user, schedule job, embedded as a shell for management and integration etc.

It has good tools for development and could utilize Java tools such as IDE and profiling tools

It provides a simplier programming model including some cutting edge technology such as AOP. (Groovy doesn't use the term AOP but interceptor)

It cohesively work with major Java libraries and framework including Spring, Hibernate, JUnit etc. Advanced things like clustering libraries makes it significantly better than other scripting and dynamic language.

Grails allows rapidly web application development.