Question 1: is the DomScape visualization possible?

In my thesis I investigate the possibility of developing a tool that visualizes any web page as a 3D model. If you don't know what I am talking about, here's a video of a prototype mock up I made using Google SketchUp.

DomScape - Prototype 1 from elbow on Vimeo.

Of course, the web page in this video is imaginary - it is however a valid example. Would such a visualization work for any web page? What are the constraining factors? Answers to these questions require extensive investigation. For now, I will try and answer them in a brief manner.

Landscaping Matryoshka's 
The two example HTML structures I used for the video were very simple. All elements contain a child element except for the top element which of course does not any other elements (it has to end somewhere). The HTML structure from the video can be compared to a Matryoshka doll, which recursively conceals similar, but smaller dolls, to be revealed once you open it.

In practice however, a web page seldom looks like that. More often than not an HTML element contains more than one 'sub-element'. Analogous to the Matryoshka doll, you would sometimes find two or more dolls concealed in a bigger doll. Like the dolls, HTML elements don't overlap in space unless they are told to do so.

The 3D representation of such a web page would look more like a landscape, in which several 'hills' are laid out over the surface. Whenever parts of the web page, such as menus, headers and footers need styling, they often become concentrations of wrapping elements. Those parts of the web page that are layered more deeply form the hills in the landscape. The more wrapping elements, the higher the hill.

The 'landscape metaphor' holds as long as all HTML elements in the web page have a static position - the default CSS positioning value. In that case, child elements stay within the boundaries of their parent elements and the web page can be thus visualized as a landscape.

Wrapping elements with position: static. Child elements never extend outside the boundaries of parent elements.

Problematic layout schemes
Other values for the position attribute are, absolute and relative, cause layout effects that are more problematic to the landscape metaphor. An element set with either value is no longer constrained by the boundaries of the parent element. The child element can extend over the boundaries of the parent element and overlap sibling elements. This results in elements occupying the same space and one element being rendered on top of the other, as is displayed in the following example.

The second element is positioned relative, making it overlap a sibling (taken from Mike Hall's article on CSS Positioning).

The CSS property overflow should also receive some consideration as to how if affects the visualization. overflow is used to define what happens when a child element contains more content than the parent has space to display. This happens for example when the parent element has a fixed height and contains too much text for that height. By setting the overflow value for the parent element, it is possible to control how the child is displayed. When overflow is set to scroll or hidden, the child element stays within the boundaries of the parent element. By default however, overflow is set to visible and this causes the child to render outside the boundaries of the parent element. How the different values for the overflow attribute are rendered is displayed below.

overflow values scroll and hidden keep the child within in the parent's boundaries. When overflow is set to visible, the content is rendered outside parent element (examples taken from the W3schools.com Tryit Editor 1.4).

The float attribute can also be used to position an element. An element can be floated to the left or the right. An element that is floated will move only horizontally, but it will also be set to extend vertically outside the parent element if the element is bigger than the parent element. This looks a lot like overflow: visible, but it's not the same. The floated element will actually influence the content in the overlapped element as if it were a child of that element. This is effect is demonstrated in the figure below.

An element is floated to the right. It extends into the the third element, which is an 'uncle' element of the floated element. That element treats the floated element a its own child and wraps the text around it (taken from Mike Hall's article on CSS Positioning).

Standard research
These positioning properties are obviously problematic to the landscape metaphor. In the last figure for example the floated element could be considered as a bridge between two elements in the landscape, but the question is how long that metaphor will be able to hold. In order to give answer to that question I will need to know exactly how these positioning properties affect the layout of web page.

The answers lie in the standard definitions that have been proposed and maintained by the World Wide Web Consortium. The W3C is a collection of representatives from important web companies and organizations (Microsoft, Mozilla (Firefox), Google, Apple, etc). They propose standards for different languages such as HTML, CSS and XML. Currently, all major browsers support HTML version 4 and CSS version 2.1.

My focus will be on the CSS2.1 specification. I will analyze the part of the standard that revolves around the positioning of elements in a web page: Section 8. Box model, Section 9. Visual formatting model and Section 10. Visual formatting model details. Mike Hall's excellent article on CSS Positioning gives a good overview of how CSS positioning works and has so far been a great help already.

Conclusion
I am investigating if a 3D visualization of a web page's underlying code is possible. Knowing exactly how CSS positioning works is only half the story. The other half is about how such positioning schemes can be visualized in 3D. It should help end users understand what the connection is between the layout and the HTML and CSS code that generated it.

Understanding how CSS positioning works might the easy part; visualizing it in a meaningful way looks dauntingly difficult at this point.

Concept video

I have created a video of the DomScape concept today. Check it out below.

DomScape - Prototype 1 from elbow on Vimeo.

Concept proposal

Please note: the following text is necessarily brief, formal and technical as it is meant to give a quick summary of my intentions for my Master thesis.

Rendering the DOM
Before a browser displays a web page, it loads HTML, Javascript and CSS. The code is interpreted and a Document Object Model is constructed. The DOM represents the data and functionality of the web page and is loaded in RAM memory so that it can be quickly accessed and manipulated. A graphical representation of the DOM is then rendered by browser’s rendering engine.

There is a set of standards defined by the W3C consortium that specify how the DOM should be rendered. The job of rendering is very hard: it needs to be fast, efficient and standards compliant. There are currently several major browsers that all differ in rendering speed and accuracy.

Understanding the rendering
The task of a web developer is to produce a certain graphical layout by using HTML, CSS and Javascript. Doing this well requires an understanding of how the rendering engine interprets the DOM. 

In reverse, understanding how a certain layout was achieved (by a third party) can be difficult as well. Because HTML and CSS are flexible, many different solutions are available to achieve a certain layout. One really needs to look at the code to be completely sure how the layout is achieved.

There are tools that make it easier to see the relationship between the code and the graphical representation. For example, Firebug enables you to inspect and compare the code and the graphical representation of that code in the browser simultaneously.

Firebug is a very handy tool for web developers, but it also has a limitation: only one element can inspected at the same time. This can be problematic in some situations. For example, when there are many wrapping elements, it can be laborious to figure out what elements contain markup and how these properties affect the layout as a whole.

I want to introduce a new way of visualizing the code in the graphical representation. This approach allows a web developer to inspect multiple elements at the same time.

A solution: visualizing the DOM hierarchy on the z-axis
The graphical representation of the DOM is confined to two dimensions. The third dimension is effectively not used in web pages (only to remove element from flow). This leaves the third dimension available to visualize the DOM hierarchy. When the graphical representation is extended to use the third dimension, the web page effectively becomes a 3D object.

The x and y positions of elements are defined by the rendering engine. The z position is deduced from the DOM hierarchy. The higher the number of ancestors, the higher the z value.

As there is distance between parent and child elements, it is easy to visually distinct between elements and what markup applies to those elements. The 3D representation is also viewable from different perspectives. This allows for an interaction in which the user can choose to focus on particular parts of the representation by looking at the representation from a certain angle and distance.

For example, when there are several wrapping elements it is hard to quickly see what element contains what markup. In a 3D representation all relevant elements and their markup can be viewed at the same time. This makes the web developer’s job easier

Small HTML snippet styled with CSS.

Same snippet, but visualized through DomScape. The DOM elements are exploded over the z-axis. Padding style is highlighted with yellow.

Plan 

There are several aspects to this thesis that need to be researched and developed. First of all, it needs a theoretical foundation. There are standards defined for both HTML and CSS. It needs to be clear on what assumptions this idea relies and whether they are right assumptions.

For example, the CSS attribute ‘overflow’ breaks the pyramidical representation. As this attribute allows an element to be bigger than its parent element, it is harder visualize in a 3D view. It is important to investigate whether there are any other CSS attributes that make a 3D visualization harder or even impossible.

Second, in order to get insight into actual use of such as visualization, it is important to do early testing with prototypes. As this is a new way of visualizing web pages, it will probably pose issues that cannot be imagined at this moment. Prototyping will help identify those issues. Usability testing with potential users will shows possible use cases and functional needs. The goal is to create a few prototypes, that demonstrate the concept.

Third, to make a 3D visualization useful it needs to offer the user more interaction and information than just that. It should visualize certain types of mark up, such as padding and margin, directly in the 3D representation. Also important are the HTML and CSS code and in what lines they apply to an element.

Fourth, it needs to be investigated how such a tool can be implemented. Ideally, the tool would function as a third party plugin to a popular browser, such as Safari, Chrome or Firefox. The tool will need access to the rendering engine to deduce x and y positions of elements, their styling properties and level of ancestry.