Canonical Ensemble

This follows the Getting Started instruction on the Apache website

My steps:

verify prerequisites:

robert@robert-linux ~ $ java -version java version "1.8.0_161" Java(TM) SE Runtime Environment (build 1.8.0_161-b12) Java HotSpot(TM) 64-Bit Server VM (build 25.161-b12, mixed mode) robert@robert-linux ~ $ python Python 2.7.12 (default, Dec  4 2017, 14:50:18) [GCC 5.4.0 20160609] on linux2 Type "help", "copyright", "credits" or "license" for more information. >>> quit()

PostgreSQL 10 has been release into production, so I wanted to give it a try in Linux Mint.  I have also installed Java 9, so I am playing just behind the bleeding edge as of Dec 12, 2017.  I installed PostgreSQL from the installer I got from the download page at EnterpriseDB.  I ran the installer, and grabbed a copy of the Pagila database (a sample database from pgFoundry) and unzipped it.  I created the database in pgAdmin 4, which is much prettier and more powerful than pgAdmin 3.  I was then able to install with pgsql using commands like for each of the 3 files in the download       /opt/PostgreSQL/10/bin/psql -U postgres -d pagila -f ~/Downloads/pagila-0.10.1/pagila-data.sql I used the order pagila-schema.sql, pagila-insert-data.sql, pagila-data.sql.

I was feeling a bit rusty on window functions, so I tried to find the top-selling movies by MPAA Rating.   The query is:

select title, rating, sales from (   select f.title,  f.rating, sum(p.amount) sales, rank() over (partition by rating  order by sum(p.amount) desc)   from film f   join inventory i using(film_id)   join rental r using(inventory_id)   join payment p using(rental_id)   group by f.film_id ) movies where rank=1 order by rating;

There are a few fun features in this query.  The joins use the ‘using’ keyword for the joins.  This rewards you for following your naming conventions by reducing typing, and I will argue, the chance of making a thinko (eg. f.film_id=i.inventory_id).  This ran on a home PC in 51ms, summing over 16,049 rows and joining 4 tables.  Nice.

The results are the 5 top selling movies by MPAA Rating.  Since the sample database defined an MPAA_Rating type, the movies ordered G, PG, PG-13, R, NC-17.  Which is what I wanted, but which you don’t get if it ordered alphabetically.

In the last chapter studied in Antonio Rangel's Principles of Economics for Scientists, we examined consumer theory.   We examined the utility function of the consumer.  In particular, we examined the difference between the experience utility function and the decision utility function.  This turns out to be a very current topic interest; Robson & Samuelson  reviewed this topic in The Evolution of Decision and Experience Utilities in Theoretical Economics 6(2011), 311-339.  As it turns out, Robson & Samuelson are two of the principals of economics for scientist.

For people who not only ‘use’ but ‘believe in’ neoclassical economics, there is almost no point in discussing the experience utility; it is the experience utility that drives economic decisions.  This mentality is what drove the ‘Mad Men’ version of marketing; a deceptive game of overselling to the gullible masses in order to amass personal wealth by artificially driving up sales.

Clay Shirky has been arguing convincingly for at least a decade that the Internet and Social Media are completely destroying this form of marketing.  Consumers are talking with consumers and these communities are upending the ability of a marketing team to broadcast a controlled message.  So the Internet and Social Media allow users to share experiences.  These shared experiences are increasingly driving the decisions of consumers.  So there is a feedback loop at play that undercuts the ability of ‘neoclassical marketing’ to ‘hype’ a product and raise the decision utility above the experience utility.  When information flows freely, you cannot maintain a differences between the demand utility and the experience utility.

This question needs a dynamic response.   Here is how I am tracking this list:

The responses to Ewa's question on LinkedIn Scala Enthusiasts, Which Companies use Scala?  In the same group, TomTom's Tom M asked How many of you are using Scala in your day job?

On Quora, What startups or tech companies are using Scala?

From the Scala language site, Scala in the Enterprise lists over a dozen major companies with short write-ups about their uses of Scala.

To keep this in perspective, Scala ranks 34 in the TIOBE software rank.

On langpop, scala ranks pretty low on the charts for usage, but is is getting a lot of buzz.

The sort answer seems to be:

finance (see analytics)

analytics (see finance)

web startups

big social sites (Twitter, LinkedIn)

Bootstrap is quickly gaining the attention of many web designers who seek faster web pages that are clean and elegant.  With Twitter engineers behind it, Bootstrap is simply a better CSS/Javascript framework than I could hope to develop.  The question that lingers is how to serve up bootstrap content.  Should this be something that I host, or should it be something that I reference from a CDN?  For Folkerts Fotografie, I have moved my content to Heroku and an using CloudFlare for DNS & as a CDN-ish service.  It is doing an good job of roughly halving the content being served by Heroku.

In the near future, I expect that CloudFlare will be using SPDY to make the 'long hauls' between their edge servers and Heroku, the image servers for Picasa the various CDNs for JavaScript, CSS, standard images and fonts used.  So that should significantly reduce the page load times.  With SPDY, the hack of referencing multiple servers to allow for parallel downloads is coming to an end.  Now, developers should be aiming to push as much as possible over a single SPDY connection.  Within a year, it should be possible to have end-to-end SPDY from Heroku/CDNs => CloudFlare => end users.

One of the great remaining pains of integrating web services is security.  The modern way is to authenticate and authorize with a combination of OpenID/OAuth, or simply to use OAuth2.  This stackoverflow discussion is a good discussion of this.  Perhaps OAuth2 is the One True Ring that will bind us all in a New World (Wide Web) Order, but probably not.  There have been several generations of authorization and authentication protocols, going back to MIT's Kerberos, which was promptly subverted by Microsoft in the 90's.  

With tools like Hadoop going mainstream for 'big data', more businesses are turing to EC2 for their analytics.  The undisputed kings of analytics on computer clusters are scientific applications like protein folding (with full quantum states determined by 'first principles', e.g. Maxwell's equations.)  So the paper, Performance Analysis of High Performance Computing Applications on the Amazon Web Services Cloud, by Jackson and co-workers is a fascinating read to see how well EC2 compares with state-of-the-art systems designed for parallel computing.  The abstract of this paper summarizes the importance of this topic:

Abstract—Cloud computing has seen tremendous growth, particularly for commercial web applications. The on-demand, pay-as-you-go model creates a flexible and cost-effective means to access compute resources. For these reasons, the scientific computing community has shown increasing interest in exploring cloud computing. However, the underlying implementation and performance of clouds are very different from those at traditional supercomputing centers. It is therefore critical to evaluate the performance of HPC applications in today’s cloud environments to understand the tradeoffs inherent in migrating to the cloud. This work represents the most comprehensive evaluation to date comparing conventional HPC platforms to Amazon EC2, using real applications representative of the workload at a typical supercomputing center. Overall results indicate that EC2 is six times slower than a typical mid-range Linux cluster, and twenty times slower than a modern HPC system. The interconnect on the EC2 cloud platform severely limits performance and causes significant variability. 

This is quite impressive to me. EC2 is within a factor of six of a dedicated cluster.  These numbers are also based upon a Sustained System Performance metric (in papers referenced in the study).  Most organizations need analytics  services with clumpy workloads, so the ability to use elastic services, rather than purchasing hardware makes a great deal of sense.

This is an example of a new generation of JavaScript libraries that are specifically aimed at web applications.  When I first started writing applications for the web, I was quite disappointed at all the fuss over N-tier applications.  The browser wasn't so much a 'client application', as it was pretty dumb terminal (parse that as you choose).  Writing a .Net web application was almost the same as writing a VB6 or MFC/C++ application.  It just forced the users somewhere across the Internet to deal with the latency of round tripping back to the web server to reach the event handler.  So a 'three tier' application felt just like the old client-server, its just that the poor end user was forced to deal with mind-numbing latency.  We developers didn't really realize this, since we were always on the same LAN (or even on the same box) when we were writing and running the applications.

Web Developers are seeing a coordinated effort to provide standard compliant browsers for HTML5, CSS3 etc.  Tools like caniuse not only help developers understand where standards are used, they also push the browser manufactuers to up their game.  But what about the world of HTML emails?  With companies like MailChimp, ExactTarget and SendGrid, we are seeing HTML emails widely used in business.  But compared with the overabundance of tools and frameworks for web developers, there is a dearth of tools for html in email.  You end up with (frequently blocked) bitmaps for icons, tables for layout - it is like waking from a flashback of 1999 and finding that it is real. (Sort of like turning on the news and seeing President Clinton in a campaign ad.)

Does anyone know of a study of what techniques can be used in HTML emails?  Where is the 'Can I Use' for MS Outlook 20xx and its ilk?  Does anyone else feel this pain?

Guess who switched to Heroku and Cloudflare on October 23 and got the performance optimization completed on October 24 (once all the DNS changes had quieted down)?  Folkerts Fotografie, that's who:-)  

Animating Bootstrap's Carousel

Bootstrap's minimal carousel is bitmap-free and versatile enough without being heavy.  But, we wanted a fade between images.  The bootstrap-carousel-animate extension looks quite promising, so its time to drop this in the carousel at Folkerts Fotografie... tomorrow.

Super simple contact page