Versions Compared

Key

  • This line was added.
  • This line was removed.
  • Formatting was changed.

...

At the same time, our choice for server-side technology needs to be scalable and forward-looking, allowing museums to invest in Engage over long run, without fear that our services will slow down impossibly when confronted with large collections, and that they won't quickly go obsolete. Engage services also need to be future-proof enough to not quickly become obsolete as the technology landscape evolves.

Lastly, the Fluid community has built up, over the years, a set of techniques and philosophies for writing software: accessibility, the Open Web, functional programming, and markup agnosticism are common themes throughout our technologies. As a community, we embrace diversity and a wide range of styles; shared patterns and techniques in code help make our solutions more coherent and consistent to develop with.

...

There is an incredible amount of diversity among museums in terms of how they each organize their collections and structure their data. Each museum's collection is different, and it is a daunting and risky task to attempt to force all types of collections into a single schema. Engage will need 's technology needs to embrace this diversity; rather . Rather than creating a "one size fits all" approach at the database level, we'll take a flexible approach to schemas in Engagetreat schemas as flexible.

Given that we can't pre-bake the data model completely, a standard SQL-based relational database in inappropriate for Engage. A new crop of document-oriented databases has emerged that are well-suited to handling dynamic schemas. Foremost among these is CouchDB, a highly scalable and Web-friendly database written in Erlang.

Couch specifically addresses the schema problem we face in Engage. Data is stored as documents in JSON format, making it particularly useful for use in client-side applications that make heavy use of JavaScript. CouchDB Views, the which provide a means for filtering, aggregating, and retrieving data from documents in the database, are written in JavaScript and can be used to make provide various data feeds from a collection of documents. This provides a solution that fits neatly within Engage's open Web architectural approach, and is approachable to a wide range of developers.

Data in CouchDB is queried and saved using a RESTful API: fetching data involves fetches use a standard HTTP GET request, while adding or updating documents, just a additions and updates use simple PUT or POST operationoperations. This means that Couch can be used with any programming language, without requiring extra native database drivers. It's also an excellent fit for a Engage's RESTful service-oriented architecture. For museums that already have their own databases or CMS systems, Couch can be replaced with a custom service that provides the same RESTful operations and feeds as provides by the Couch Views that will ship out of the box with Engage.

...

Users have come to expect natural, Google-like text searches from most modern Web experiences. As a result, Engage will need to include a free text search engine service.

Hands down, the most viable best open source technology for this is Apache's Lucene. Today, it's the fastest and most reliable open source text searching software available, and it has few competitors. Lucene is a JVM-based tool, and should be relatively easy to install in most server environments. Lucene can be easily connected to CouchDB, allowing new documents in Couch to seamlessly be added to the indexes.

...

At the same time, JavaScript is probably the most widely used and understood programming language today. Regardless of what technologies they use are used on the server-side, the majority of Web developers also write client-side code in JavaScript. In terms of the ubiquity angle, JavaScript is overwhelming familiar and accessible to programmers coming from diverse background and technology camps.

...

JavaScript on the server, while still on the bleeding edge, provides will provide us with an broadly approachable environment for writing server-side code using many of the existing tools offered by Fluid Infusion.

...

Currently, the most mature and reliable environment for running JavaScript on the server is Mozilla's Rhino. Running with within a Java Virtual Machine, Rhino is used widely for embedded scripting in many Java applications, and also ships as part of the standard Sun Java 6 JDK. As a result, Rhino is widely tested and stable.

...

Rhino's deployment within the JVM brings with it a number of advantages. We can leverage the Servlet API to provide us with a proven interface to Apache and other Web servers. Since threading in Java is well-establishestablished, and the Servlet API offers clear semantics for concurrency, the JVM offers an easy way to establish a container and framework that is scalable and concurrency-safe.

...

In the past, JavaScript has been regarded as a slow scripting language. With the browser wars back in full swing, new JavaScript runtimes have emerged in the latest releases of every major browser. Even more than any other scripting language, JavaScript VM optimisation is receiving very significant attention these days.

Google's V8 engine and Mozilla's TraceMonkey engines are both blazingly fast, bringing JavaScript performance at least up to levels comparable to other dynamic languages such as Ruby, Python, and PHP.

The long term roadmap for the Engage services layer will be to move from Rhino on the JVM to one of these next-generation JavaScript runtimes. While there are Web server adaptors for both these runtimes available today—most prominently v8cgi—we expect it will take a number of months before these are production-ready. As soon as we're confident in the concurrency, cross-platform support, and reliability of using a next-generation runtime for server-side Web development, we'll switch. In the meantime, Rhino and the JVM will provide us with a stable and acceptable production environment.

Portability and JSGI

With a this plan to build first on Rhino and the JVM, and then move to a next-generation runtime as soon as the required Web infrastructure is established for one of thesethey're ready, the obvious problem is portability. If we commit to Rhino now, will be able to move to V8 or Tracemonkey later?

The JSGI spec provides a solution to the portability issue. Modeled after Python's WSGI and Ruby's Rack, JSGI provides a simple contract for Web containers when managing , specifying how requests and responses should be handled. JSGI is well-suited to JavaScript's functional style, and is relatively straightforward to implement. By coding to JSGI, the Engage server-side technology can be trivially ported from Rhino + Servlets to v8cgi or another next-generative runtime.

...