Asset Management, the Status Quo and Areas Of Opportunity

Why do we organize assets the way we do? It must be rooted in tradition.

Back when the internets were first getting started, and the world wide web was at version “1.0”, we just didn’t have that many .css or .js files to organize. So we put all our .css files in a public directory called “assets/css” and all our .js files in a public directory called “assets/js”, and we were done.

Fast forward 15 years to “think client” web applications built on JavaScript MVC frameworks. Now we have multitudes of little .js and .css snippets that relate to each other (e.g. personModel.js, personView.js, person.css, etc.), as well as client side templates thrown into the mix that relate to specific .js and .css files.

Yet most of us are still using the same traditional directory structure, grouping all of our assets by type of asset, even though that directory structure provides little-to-no value. Why don’t we leverage God’s gift of the directory a bit more effectively to put assets together that really belong together? For example, instead of having the “person” related assets spread out all over the place, why don’t we just put all of those assets into one directory?

It sure would be nice to be able to quickly switch between “person” files just by clicking on another file in the same directory! And what about assets that are used just by one particular page? Why don’t we put those assets in the same directory as the template for that page, since all those files are so closely related?

What’s the holdup?

Nice thought, hard to materialize. The problem is that asset management has a lot of moving parts. A complete solution needs to address preprocessing (i.e. compiling .scss, .coffee, etc.), minification and concatenation in production mode, and of course, dependency management. Dependency management becomes a real pain in multi-page applications. Although there are partial solutions available, I am not aware of any existing solution that handles all of these issues and all three types of asset files (.js, .css, and templates).

Thought experiment

But let’s not give up hope. In a perfect world, what would your ultimate asset manager look like? Here are some of the features we came up with.

• Assets grouped together in a directory would define a “bundle”, containing related javascript, styles, and templates.
• Bundles could have dependencies on other bundles.
• Pages could have dependencies on bundles, and could also use additional “private” assets that live in their directory.
• All assets that a page uses would be automagically injected into the served HTML when the page’s template is rendered. No more futzing with <script> and <link> tags!
  • In development mode, served assets should be preprocessed, but not minified or concatenated.
  • In production mode, served assets should be preprocessed, minified and concatenated.
• Assets that are not used by a particular page should NOT be served with that page.
• The developer should not be forced to use a particular JavaScript module system (e.g. AMD, CommonJS).

What do you think? How would you manage your assets if you could snap your fingers and make it so?

Existing solutions

There are several tools in this general space, some of them look confusingly similar.

RequireJS together with its optimizer can manage dependencies well, minify and concatenate your JavaScript, and it can also handle templates with the text plugin. However, support for stylesheets is limited (no built in precompilation, concatenation or minification) and buggy. Managing assets for multi-page applications is also challenging, and, of course, you are obligated to use to the AMD module system.

The Rails Asset Pipeline preprocesses, minifies, and concatenates assets in RoR applications. Assetic for PHP fills the same niche in the PHP world. You list each individual asset that is required by a given page, and then they are automatically served up un-minified, individually in development mode but minified and concatenated in production mode. Templates are not supported, and javascripts and stylesheets must be dealt with totally separately. There is no dependency management. Although you can use any directory structure you want with these tools, because it is cumbersome to ensure that the proper assets are included on each page in the correct order, most people just stick to the traditional model of making a folder for each of their asset types, and then just include them all.

GruntJS is a build tool that allows you to many automate repetitive tasks including preprocessing, minification and concatenation of assets. It is a heavy lifter but it is rather clumsy and brainless. It offers no easy way to select the appropriate assets to process for a given page, no dependency management, and, since it is just a build tool, no means to include assets into a page once they have been processed.

Bower is a package manager that fetches asset from online sources like GitHub, taking dependencies into account. It does not handle preprossing, minification, concatenation, or integration of assets into web applications. Bower is geared towards package management, or fetching assets, not building or serving them.

Component is similar to and competes directly with Bower, but also concatenates and serves .css and .js files. However there is no built in support for preprocessing assets, or switching between development and production environments. It also requires the use of its CommonJS implementation, and has poor support for multi-page applications.

Although these are powerful tools, they don't hit all the items on our wish list. We have built applications using Assetic with the directory structure outlined at the beginning of this post. The directory structure itself is very helpful, but there are several pain points (mentioned above) that get worse as your application scales. Developing a solution that works better for our needs is one of our priorities. We’d love to hear your feedback on what your ideal asset manager would look like. If you want to keep up to date on our findings and progress, get in touch.

UPDATE:

An open source tool that addresses the needs described in this post is being developed! Check out the Cartero repository on GitHub for the latest.