robertfolkerts-blog

[robertfolkerts-blog]

Installing Cassandra

This follows the Getting Started instruction on the Apache website

My steps:

verify prerequisites:

robert@robert-linux ~ $ java -version java version "1.8.0_161" Java(TM) SE Runtime Environment (build 1.8.0_161-b12) Java HotSpot(TM) 64-Bit Server VM (build 25.161-b12, mixed mode) robert@robert-linux ~ $ python Python 2.7.12 (default, Dec  4 2017, 14:50:18) [GCC 5.4.0 20160609] on linux2 Type "help", "copyright", "credits" or "license" for more information. >>> quit()

Then, I installed Cassandra following the Debian instructions.  Reading the configuration page, 

[robertfolkerts-blog]

Damien provides a careful examination of how to use Akka-Persistence and Cassandra within an Event Sourcing model.  A look at his blogs show that there is a lot of detailed content on Scala related topics.

[robertfolkerts-blog]

PostgreSQL 10 - kicking the tires

PostgreSQL 10 has been release into production, so I wanted to give it a try in Linux Mint.  I have also installed Java 9, so I am playing just behind the bleeding edge as of Dec 12, 2017.  I installed PostgreSQL from the installer I got from the download page at EnterpriseDB.  I ran the installer, and grabbed a copy of the Pagila database (a sample database from pgFoundry) and unzipped it.  I created the database in pgAdmin 4, which is much prettier and more powerful than pgAdmin 3.  I was then able to install with pgsql using commands like for each of the 3 files in the download       /opt/PostgreSQL/10/bin/psql -U postgres -d pagila -f ~/Downloads/pagila-0.10.1/pagila-data.sql I used the order pagila-schema.sql, pagila-insert-data.sql, pagila-data.sql.

I was feeling a bit rusty on window functions, so I tried to find the top-selling movies by MPAA Rating.   The query is:

select title, rating, sales from (   select f.title,  f.rating, sum(p.amount) sales, rank() over (partition by rating  order by sum(p.amount) desc)   from film f   join inventory i using(film_id)   join rental r using(inventory_id)   join payment p using(rental_id)   group by f.film_id ) movies where rank=1 order by rating;

There are a few fun features in this query.  The joins use the ‘using’ keyword for the joins.  This rewards you for following your naming conventions by reducing typing, and I will argue, the chance of making a thinko (eg. f.film_id=i.inventory_id).  This ran on a home PC in 51ms, summing over 16,049 rows and joining 4 tables.  Nice.

The results are the 5 top selling movies by MPAA Rating.  Since the sample database defined an MPAA_Rating type, the movies ordered G, PG, PG-13, R, NC-17.  Which is what I wanted, but which you don’t get if it ordered alphabetically.

So, I was please that it installed and ran with little hassle.

#postgresql #sql #enterprisedb #pagila #pgFoundry

[robertfolkerts-blog]

High performance web applications with websockets

Frank Greco, Kaazing's Diector of Technology, gave a talk, High Performance Messaging for Web Applications, at QCon London 2013.  This talk nicely explains how Websockets should be be used - as a foundation for higher level protocols.  Hardly anyone still programs with IP sockets, we program at a higher level - like JMS, AMQP, etc.  Websockets should be the same.  There is also JSR 356, the Java API for websockets.  What is as interesting was the discussion of JMS over websockets.  He then shows JMS over web sockets.  I enjoyed his speech and it did show me how web sockets will change web programming.

#websockets #QCon #Frank Greco #Kaazing

[robertfolkerts-blog]

Marketing and Experience Utility

In the last chapter studied in Antonio Rangel's Principles of Economics for Scientists, we examined consumer theory.   We examined the utility function of the consumer.  In particular, we examined the difference between the experience utility function and the decision utility function.  This turns out to be a very current topic interest; Robson & Samuelson  reviewed this topic in The Evolution of Decision and Experience Utilities in Theoretical Economics 6(2011), 311-339.  As it turns out, Robson & Samuelson are two of the principals of economics for scientist.

For people who not only ‘use’ but ‘believe in’ neoclassical economics, there is almost no point in discussing the experience utility; it is the experience utility that drives economic decisions.  This mentality is what drove the ‘Mad Men’ version of marketing; a deceptive game of overselling to the gullible masses in order to amass personal wealth by artificially driving up sales.

Clay Shirky has been arguing convincingly for at least a decade that the Internet and Social Media are completely destroying this form of marketing.  Consumers are talking with consumers and these communities are upending the ability of a marketing team to broadcast a controlled message.  So the Internet and Social Media allow users to share experiences.  These shared experiences are increasingly driving the decisions of consumers.  So there is a feedback loop at play that undercuts the ability of ‘neoclassical marketing’ to ‘hype’ a product and raise the decision utility above the experience utility.  When information flows freely, you cannot maintain a differences between the demand utility and the experience utility.

This leads me to this conclusion:  The role of marketing in the Internet Age is to share with consumers the true experience utility of their products.  Furthermore, it is it allow the voice of the consumer to become integrated with product design in order to increase the experience utility of the product.  This is how you drive up a sustainable demand for your product.

#Paul Samuelson #Clay Shirky #experience utility #demand utility #Antonio Rangel #Arthur Robson #marketing

[robertfolkerts-blog]

If I can't parallelize it, it is probably a syntax error

Cliff Click, from Big Data, Small Computers at Strangeloop 1012

#Cliff Click #JVM #Scala

[robertfolkerts-blog]

Who uses Scala?

This question needs a dynamic response.   Here is how I am tracking this list:

The responses to Ewa's question on LinkedIn Scala Enthusiasts, Which Companies use Scala?  In the same group, TomTom's Tom M asked How many of you are using Scala in your day job?

On Quora, What startups or tech companies are using Scala?

From the Scala language site, Scala in the Enterprise lists over a dozen major companies with short write-ups about their uses of Scala.

To keep this in perspective, Scala ranks 34 in the TIOBE software rank.

On langpop, scala ranks pretty low on the charts for usage, but is is getting a lot of buzz.

The sort answer seems to be:

finance (see analytics)

analytics (see finance)

web startups

big social sites (Twitter, LinkedIn)

Scala has a strong following in Europe (Switzerland, Paris, London, Benelux seen well represented).  In the US, the interest seems to be centered in Silicon Valley/San Francisco, New York and Boston.  These is also some real growth in Latin America and Australia.  

#Scala

[robertfolkerts-blog]

Bootstrap and CDNs

Bootstrap is quickly gaining the attention of many web designers who seek faster web pages that are clean and elegant.  With Twitter engineers behind it, Bootstrap is simply a better CSS/Javascript framework than I could hope to develop.  The question that lingers is how to serve up bootstrap content.  Should this be something that I host, or should it be something that I reference from a CDN?  For Folkerts Fotografie, I have moved my content to Heroku and an using CloudFlare for DNS & as a CDN-ish service.  It is doing an good job of roughly halving the content being served by Heroku.

In the near future, I expect that CloudFlare will be using SPDY to make the 'long hauls' between their edge servers and Heroku, the image servers for Picasa the various CDNs for JavaScript, CSS, standard images and fonts used.  So that should significantly reduce the page load times.  With SPDY, the hack of referencing multiple servers to allow for parallel downloads is coming to an end.  Now, developers should be aiming to push as much as possible over a single SPDY connection.  Within a year, it should be possible to have end-to-end SPDY from Heroku/CDNs => CloudFlare => end users.

In this world, I want a small number of CDNs that will serve up as much 'standard content' as possible. BootstrapCDN seems ideal for bootstrap while cdnjs seems ideal for most of the jQuery-related content.  

[robertfolkerts-blog]

Sacha Labourey: The Cloud as a Tectonic Shift in IT.mp4 (by CloudBeesTV).  Sacha does a great overview of the XaaS environment and shows where CloudBees fits in.  With tools like CloudBees for a Jenkins provider, the Play framework for application framework, Akka and Apache Camel for web services, and Symplified for security, developers can call best-of-breed providers for the services that are not strategic  to their applications.

[robertfolkerts-blog]

Single Sign-on: Symplified

One of the great remaining pains of integrating web services is security.  The modern way is to authenticate and authorize with a combination of OpenID/OAuth, or simply to use OAuth2.  This stackoverflow discussion is a good discussion of this.  Perhaps OAuth2 is the One True Ring that will bind us all in a New World (Wide Web) Order, but probably not.  There have been several generations of authorization and authentication protocols, going back to MIT's Kerberos, which was promptly subverted by Microsoft in the 90's.  

Understanding and including authentication/authorization from various competing camps and various generations of tools, is leech on developer productivity.  Symplified bills itself as The Cloud Identity Platform.  I sure hope this works well, because it sure would be nice to have an expert manage a user ID chain and security services.

[robertfolkerts-blog]

The IBM Institute for Business Value surveyed more than 1,100 executives and conducted in-depth interviews with 26 companies worldwide. This study shows how leading companies are integrating social into their strategies to:

Create valued customer experiences

Drive workforce productivity and effectiveness

Accelerate innovation

Read all the results and discover how well organizations are facing the opportunities and challenges.

[robertfolkerts-blog]

The Mobull modular data center at University of Perpignan Via Domitia.  Using this data center really is like drinking from the fire hose. The data center is water cooled and the connections to the water mains are much larger than the connections to the electrical mains.  There are similar mobile data centers from other manufactures, but it's pretty hard  to miss the double entendre of Mobull.

#bullx #mobull

[robertfolkerts-blog]

Lawrence Berkeley report on EC2 performance

With tools like Hadoop going mainstream for 'big data', more businesses are turing to EC2 for their analytics.  The undisputed kings of analytics on computer clusters are scientific applications like protein folding (with full quantum states determined by 'first principles', e.g. Maxwell's equations.)  So the paper, Performance Analysis of High Performance Computing Applications on the Amazon Web Services Cloud, by Jackson and co-workers is a fascinating read to see how well EC2 compares with state-of-the-art systems designed for parallel computing.  The abstract of this paper summarizes the importance of this topic:

Abstract—Cloud computing has seen tremendous growth, particularly for commercial web applications. The on-demand, pay-as-you-go model creates a flexible and cost-effective means to access compute resources. For these reasons, the scientific computing community has shown increasing interest in exploring cloud computing. However, the underlying implementation and performance of clouds are very different from those at traditional supercomputing centers. It is therefore critical to evaluate the performance of HPC applications in today’s cloud environments to understand the tradeoffs inherent in migrating to the cloud. This work represents the most comprehensive evaluation to date comparing conventional HPC platforms to Amazon EC2, using real applications representative of the workload at a typical supercomputing center. Overall results indicate that EC2 is six times slower than a typical mid-range Linux cluster, and twenty times slower than a modern HPC system. The interconnect on the EC2 cloud platform severely limits performance and causes significant variability. 

This is quite impressive to me. EC2 is within a factor of six of a dedicated cluster.  These numbers are also based upon a Sustained System Performance metric (in papers referenced in the study).  Most organizations need analytics  services with clumpy workloads, so the ability to use elastic services, rather than purchasing hardware makes a great deal of sense.

If we look at numbers in the report, it is pretty clear that the latency of the EC2 gigabit either net is a big cause of the performance limits.  So, if Amazon (or any other cloud provider) were to use an InfiniBand system, like the Bullx system, the latency (between nodes) would plummet.  Besides, what technology geek can look at the Mobull system and not smile? Look at this, you can have a data center that is literally drinking from the firehose!

[robertfolkerts-blog]

This is an example of a new generation of JavaScript libraries that are specifically aimed at web applications.  When I first started writing applications for the web, I was quite disappointed at all the fuss over N-tier applications.  The browser wasn't so much a 'client application', as it was pretty dumb terminal (parse that as you choose).  Writing a .Net web application was almost the same as writing a VB6 or MFC/C++ application.  It just forced the users somewhere across the Internet to deal with the latency of round tripping back to the web server to reach the event handler.  So a 'three tier' application felt just like the old client-server, its just that the poor end user was forced to deal with mind-numbing latency.  We developers didn't really realize this, since we were always on the same LAN (or even on the same box) when we were writing and running the applications.

But, we are finally getting clients that run in the browser.  Timeline JS is a lovely way to present timelines in a browser.  This caught my eye because I wanted to present Atom feeds that show a chronology of an entity's evolution (using entity in the sense of domain driven design).  As noted by the Night News Innovation Lab at Northwestern, journalists around the world are adopting Timeline JS to use in their on-line media.  So this is one to watch.

#Timeline JS

[robertfolkerts-blog]

HTML Email Clients: why is support so crappy?

Web Developers are seeing a coordinated effort to provide standard compliant browsers for HTML5, CSS3 etc.  Tools like caniuse not only help developers understand where standards are used, they also push the browser manufactuers to up their game.  But what about the world of HTML emails?  With companies like MailChimp, ExactTarget and SendGrid, we are seeing HTML emails widely used in business.  But compared with the overabundance of tools and frameworks for web developers, there is a dearth of tools for html in email.  You end up with (frequently blocked) bitmaps for icons, tables for layout - it is like waking from a flashback of 1999 and finding that it is real. (Sort of like turning on the news and seeing President Clinton in a campaign ad.)

Does anyone know of a study of what techniques can be used in HTML emails?  Where is the 'Can I Use' for MS Outlook 20xx and its ilk?  Does anyone else feel this pain?

There are others who have had the same issue at stackoverfow: Testing Email Rendering gives some tools for testing.  There is a commercial Email On Acid service that helps with testing and optimizing; they have a free service, but I haven't tried it (yet).

#HTML email #MS Outlook #caniuse #MailChimp #ExactTarget #SendGrid

[robertfolkerts-blog]

Guess who switched to Heroku and Cloudflare on October 23 and got the performance optimization completed on October 24 (once all the DNS changes had quieted down)?  Folkerts Fotografie, that's who:-)  

These numbers, as reported by Pingdom, are an almost unbelievable improvement.  The performance over the previous app engine platform is really dramatic.  To say that we are happy with the combo of Play Framework 2.x, Heroku and Cloudflare is quite the understatement.

#Cloudflare #Heroku #Folkerts Fotografie #Pingdom #Play Framework 2.0

[robertfolkerts-blog]

Playing with Bootstrap

Animating Bootstrap's Carousel

Bootstrap's minimal carousel is bitmap-free and versatile enough without being heavy.  But, we wanted a fade between images.  The bootstrap-carousel-animate extension looks quite promising, so its time to drop this in the carousel at Folkerts Fotografie... tomorrow.

Super simple contact page

I want to make the contact us page have one sentence and 4 huge icons.  With tooltipser, each big icon can have a rich (html content) tooltip.  With bootstrap-modal, the form will disappear until the user clicks the form icon.  If I get ambitious, the icons can use a css animation on page load to highlight each one in order to draw your eye across the different contact options for the user.

#Bootstrap 2.0 #Folkerts Fotografie