Nodir Turakulov

gRPC is great, but is not available on Classic AppEngine at this time, so while waiting for it we wrote pRPC (provisional RPC) for Go. It is compatible with gRPC server/client code, but works on HTTP 1.x and Classic AppEngine. In addition it has gRPC-compatible discovery and a CLI tool.

IDL

Just like gRPC, pRPC uses Protocol Buffers to define API. I assume you know why protobufs are awesome because you care about gRPC.

This definition is taken from gRPC examples. pRPC proto definition is same as gRPC, but pRPC does not support streams.

Compile .proto

For Go code, we had to slightly alter (in a backward-compatible way) the code generated by protocol buffer compiler, so we wrote cproto tool.

Install cproto:

go get -u github.com/golang/protobuf/{proto,protoc-gen-go} go get -u github.com/luci/luci-go/grpc/cmd/cproto

cproto has minimalistic CLI:

//go:generate cproto

compiles all .proto files in the current directory to .go files

includes $GOPATH/src in proto import path, so you can (and should) import other .proto files with Go-style absolute paths, e.g. import "github.com/user/repo/proto/something.proto";

generates pb.discovery.go file that registers full service description for discoverability. You don't have to worry about it, it is just there and it is optional.

supports only Go

OK, now we have compiled our .proto files to Go code. Let's implement a service!

Service

A service implementation is fully compatible with gRPC:

pRPC uses gRPC codes

HTTP headers are accessible through metadata in the context.

In the code snippets I omit imports, but you can find them in the source code.

Server

Now let's host the service. Since AppEngine is our target user case, the example is for AppEngine. The init function registers HTTP routers in the default muxer.

This example registers pRPC HTTP handlers for helloworld service.

I've deployed prpc-helloworld.appspot.com, but you can run your own server locally:

goapp get -u github.com/nodirt/prpc-example/helloworld/server goapp serve github.com/nodirt/prpc-example/helloworld/server

Assuming your $GOPATH is correctly configured for goapp, this will run a pRPC server locally at port 8080.

CLI client

rpc is a CLI tool that can discover services and call them.

Install rpc:

go get -u github.com/luci/luci-go/grpc/cmd/rpc

I will use prpc-helloworld.appspot.com for future examples (and so can you), but you can use :8080 for your local server.

Discover services

List available services:

$ rpc show prpc-helloworld.appspot.com helloworld.Greeter discovery.Discovery

List service methods:

$ rpc show prpc-helloworld.appspot.com helloworld.Greeter // The greeting service definition. service Greeter { // Sends a greeting rpc SayHello(HelloRequest) returns (HelloReply) {}; }

Describe a service method:

$ rpc show prpc-helloworld.appspot.com helloworld.Greeter.SayHello // The greeting service definition. service Greeter { // Sends a greeting rpc SayHello(HelloRequest) returns (HelloReply) {}; } // The request message containing the user's name. message HelloRequest { string name = 1; } // The response message containing the greetings message HelloReply { string message = 1; }

Call services

rpc can make RPCs:

$ rpc call prpc-helloworld.appspot.com helloworld.Greeter.SayHello -name $USER message: "Hello nodir"

Go client

Now let's make RPCs ourselves. A pRPC client implements the same interface as gRPC client, but it is created using a different function:

This may print a gRPC error code to stderr or the greeting to stdout.

Try it yourself:

go get -u github.com/nodirt/prpc-example/helloworld/client/prpchello prpchello -server prpc-helloworld.appspot.com -name $USER

This should greet you.

gRPC compatibility

Server and client interfaces for servies are same for pRPC and gRPC.

The discovery is just a service that works with both gRPC and pRPC, but it relies on pb.discovery.go files generated by cproto.

The rpc tool currently works with pRPC only, but we will update it when we switch to gRPC. Patches are welcome too.

Authentication

Unlike gRPC, authentication in pRPC is based on HTTP because our main use case is OAuth.

Our repo implements GAE-based OAuth2. To enable it, all you need to do is to import github.com/luci/luci-go/appengine/gaeauth/server package and use our authentication framework, e.g. use CurrentIdentity function to read current user email.

Alternatively, you can set prpc.Server.CustomAuthenticator to true and do whatever you want in base function passed to Server.InstallHandlers.

RPC Explorer

With RPC Explorer you can discover services, methods, with documentation, and make requests. Has request autocompletion.

Demo: https://prpc-talk.appspot.com/rpcexplorer/

Protocol

The Go stdlib has a struct log.Logger. You may be wondering why it is not an interface and here is why.

Before we proceed, let's assume for this post that log.Logger's API is fine in spite of lack of logging level.

In traditional programming languages, such as Java, the stdlib defines canonical interfaces that others can implement. Go stdlib does that too, but only when it is necessary, and the necessity is determined by whether the interface is consumed by the stdlib. This is because Go interfaces are implemented by types implicitly.

If you define MyLogger interface with the log.Logger methods you actually need, log.Logger automatically implements it.

If a third-party library author wants to define an alternative implementation for an existing std type, such as log.Logger, all she needs is to conform its API (methods). Then their implementation automatically implements your MyLogger too.

Now instead of accepting *log.Logger parameters in your functions, you can always accept the MyLogger interface. Both log.Logger and better.Logger can be used as an argument.

It is a good idea to accept an interface as a parameter, rather than a concrete type. This way you allow callers to pass any implementation.

Usually my mind is so stressed that I'm unable to concentrate on anything. My attention is chaotic. Background thoughts, such as reflection and doubts, constantly get in a way of foreground thoughts, which affects my outer interaction (cognition, speech, reactions, etc). In particular, it makes it hard for me to effectively consume information: not much of what I read in a book actually gets into my mind; as result I fall asleep in 5 min after starting to read in bed. When I talk to a person, I may be thinking of something else at the same time, and I have only one brain so it negatively affects the quality of the conversion; I don't always have something to respond because I might be thinking of something else. I'm not proud of it.

So today I tried to practice mindfulness by just watching a fountain in our apartment community. Apparently it is hard to keep the attention on the water flows and not to think of anything else. Attention kept flying away to matters of my concerns, fears, doubts, etc; and I kept moving it back to the fountain.

Then an interesting thing happened. Just as Chad-Meng said in his talk, the resolution of the picture has increased as I kept concentrating on the present moment. I started to distinguish sounds of water coming out of the pipe from the sounds of the falling water. They are actually deferent because water comes from the pipe evenly but falls non-evenly. I also became calmer.

Tonight, for the first time for a long period of time, I didn't fall asleep while reading a book in bed. The book was Search Inside Yourself by a mindful Googler. I think the quality of the information ingestion increased too: background thoughts mostly didn't get in a way. Sometimes they did and I deliberately brought my attention back to the book, and reread the paragraph so nothing is missed.

In case you need to read from both stdout and stderr of a subprocess in Go, line by line:

TL;DR Read three books, watch two classes, solve problems at topcoder/codeforces/projecteuler. Links are below.

If you are interested in getting a job at Google, you've probably already read Steve Yegge's famous post (if you didn't, read it first). It assumes you have only a couple of weeks for preparation and focuses on topics. I used it as a checklist. This recipe focuses on resources; depending on your background, requires 3-6 months, and gives you pretty good chances to pass the interview even if you never knew the difference between a graph and a hash table.

After following the recipe you will

have basic math, algorithm and data structure knowledge required for the interview

change the way you think about problems, expressing them using mathematical models (it is not scary, rather fun)

possibly love learning. Here at Google it is a continuous process and not less intensive.

learn nothing about system design (see below)

Four months ago I complained that there is no Tashkent Developer Community. So here is an update. Today is one month since I've created group tasdev on Facebook. We have now 250 registered members where about 20 are active online, and about 40-70 attended 5 meetups, maybe more.

Today I've discovered, thanks to a customer, that two weeks ago ComponentOne Studio® for Entity Framework, the main product I worked on, won Code Project Members Choice 2012 award in the Big Data Solutions category at Microsoft’s TechEd 2012 conference in Orlando.

This award is the best thing I ever achieved and thus I am super excited. Such awards are a great source of motivation.

Posted on June 26, 2012

Update: After learning how others do class inheritance I've learned that this approach adds nothing new. John Resig has already invented it.

Basic classes

Classic tutorials teach us to define classes this way:

function Person(name) { this.name = name; } Person.prototype.greeting = function (pleasant) { console.log("Hi. I'm " + this.name); if (pleasant) { console.log("How are you?"); } }; Person.prototype.sleep = function () { // complex algorithm };

But why not to define classes like this:

var Person = defineClass({ constructor: function (name) { this.name = name; }, greeting: function (pleasant) { console.log("Hi, my name is " + this.name); if (pleasant) { console.log("How are you?"); } }, sleep: function () { // complex algorithm } }); var person = new Person("John"); person.greeting(true);

Many developers, I am included, are used to write code that looks like this:

class Foo { Baz _bar; public Baz Bar { get { return _bar; } set { if (Equals(_bar, value)) return; _bar = value; // some code, e.g. notifications } } }

Let me blame C# again. It doesn't allow to have any statements in a constructor before a call a base/this constructor:

class Foo { // suppose you can't change this signature public Foo(int x, int y) { ... } } class Bar : Foo { public Bar(Point p) : base(p.X, p.Y) { } // call of the base constructor } class Point { public int X, Y; }

however, this way you can't do some things with your constructor arguments in a usual way, for example check some pre-conditions, or perform some calculations before calling the base constructor. The p argument above could be null, and that would cause an exception.

Existing solution

A partial solution is to put calculations/checks right into the base/this constructor call. Often a part of this code has to be extracted into a separate static method.

class Bar : Foo { static Point CheckNull(Point p) { if (p == null) throw new NullReferenceException("p"); return p; } public Bar(Point p) : base(CheckNull(p).X, p.X) { } }

but it's ugly. A simple pre-condition check turned into a separate method.

Won't work

Anyway that doesn't work in all cases. For example, you can't use the same static method call for multiple parameters of the base/this constructor:

class Bar { static Point SlowMethod(Point p) { ... } public Bar(Point p) : base(SlowMethod(p).X, SlowMethod(p).Y) { } // SlowMethod is called twice }

Also it won't work if you want to use Code Contracts.

Possible solution: the Java way

If C# allowed you to insert statements before a base/this constructor call, the code above would look like this:

class Bar { Point SlowMethod(Point p) { ... } public Bar(Point p) { p = SlowMethod(p); base(p.X, p.Y); } }

I understand that it is unsafe to touch the this object before the base/this constructor is called, but C# Team could easily just prohibit any access to this as they do in the base/this constructor argument expressions.

Possible solution: functional

A functional solution is to

Make every method receive no more than one parameter. If you want to have multiple parameters, then they are converted to a single tuple parameter.

Make every statement an expression (like in Scala). This way you can write ifs, variable declarations, block statements, throw exceptions and so on as a a part of an expression.

class Foo { // think of these two parameters as of // a single parameter of type Tuple<int, int> public Foo(int x, int y) { ... } } class Bar : Foo { public Bar(Point p) : base({ if (p == null) throw new ArgumentNullException("p"); (p.X, p.Y) }) { } }

The point of optional parameters is to abstract from them unless you need to pass some special argument values.

void Foo(bool bar = false) { ... } ... Foo(); Foo(true); // specify args on demand

However C# compiler fails on this code:

Action foo = Foo; // compilation error :(

which is sad. IMO it should handle this case. I have to write

Action foo = () => Foo();

Before meeting Scala, I though that C# is a very powerful PL.