ToDos__14_Jy_02.txt

#-] Create skeleton of system
#- -] create an app module that acts as test scaffolding
#- -] create a Display module and a visualizer module
#- -] in the app, create the display and visualizer and connect them
#- -] in the app, create the GabePattern syntax graph inside the srcHolder, and set that to be what gets visualized.
#- -] in app, trigger the visualizer to generate a view hierarchy and send it to the display, which in turn triggers display to generate famous render tree.

//====== Visualizer ======
//=
//========================
-] make a function that accepts a native syntax graph and builds a view hierarchy from it, then hands that view hierarchy data structure to the Display
#- -] First, make the prototype, with skeleton of internal structure
- -] second, do by-hand code that creates the view hierarchy of the two-box visualization of part of the syntax graph of the GabePattern example


//====== Display ======
//=
//========================
-] make a function that accepts a view hierarchy data structure and builds a famous render tree from that
#- -] first, make the prototype and skeleton of internal structure


=====================
# -] So, the task at hand is to draw a single syntax graph element, by using famous.  After that, add transforms that change the size.. then ones that shift positions.. and so on, go from there..  until have a syntax graph drawn.  
# - -] Try taking an SVG representation of the round-cornered box and converting that to whatever famous takes as its native vector graphic/drawing format.  Saw a googled thing about a converter from svg to something famous understood, like some form of opengl or something.

# -] Get SVG working with famous -- draw the sigma example, hard-coding it, with SVG copied from inkscape output.

#-] Q: how to remove a node from the render tree?  Or from a context surface?
-[] A: CAN'T!!!!!  Have to regenerate the whole damn thing!

#-] Get famous working without a net connection -- download whatever URLs that the page references, so are local files.

XXX -] Get famous fonts -- download to local somehow..

#-] Figure out how that other graph drawing thing made the curvy arrow-lines between nodes..

#-] Get bezier curves in SVG working on syntax graph page, going between text surface and middle of target box side, with control points set to mix x of one end point with y of other end point

XXX -] Next, draw two element nodes, complete with properties and ports, with linked ports -- so know correct code, that is known to work. [-] don't need to go this far, all the essential elements have been hit.

#-] Look back at Java stuff, see what were doing for structure sent from visualizer to Display
[-] Were also doing a hierarchy of bounding boxes
[-] Were parsing the syntax graph -- generate a bounding box and visual contents for each kind of element in the syntax graph

#-] Next, define a data structure for the visual elements.
#- -] within it is the information needed to construct the graph that have drawn
#- -] see below for visual element design.. and #look back over java version..

#-] Next, take the syntax graph for the GabePatterns, create the visual elements to represent that.

#-] Next, write the Display object to receive the visual elements, and then create the famous objects from it, which end up painting the syntax graph elements as seen in the PDF.

#-] Next, create the visual element objects for the rest of the syntax graph, and have them all draw to the screen, using famous.

#-] Make a Display object, which contains the code wrote above.

#-] Have the Display object receive the visual element object, and proceed to draw it to the screen using famous.  Choose a protocol for handing visual element objects to the Display and triggering the Display to render them via famous.

#-] Make the visual elements for a graph of 5 nodes.  hand them to the display.

XXX -] Make a "change" visual element.  Have the display modify its famous structure according to the change visual element.
[-] Famous can't do this!  Just send a new visual hierarchy and render the whole thing

//================  Visualizer =================

-] Separately, the task is to construct a visual hierarchy structure inside the visualizer..  then to convert that into visual elements..  #( this part  done) receive those elements in the famous display, and reconstruct a famous structure from the stream of visual elements.

#-] Create a source holder object.. just a simple thing, as a place holder

-] dump the syntax graph wrote for Gabe out to a JSON file

-] have the source holder object load the JSON in, and convert to the javascript objects wrote for Gabe

-] Then, make a new object, just one syntax graph element, as above.  Send it to a JSON file.  Have the source holder read it in and convert to javascript object.

#-] make a visualizer object inside the source holder

#-] give the visualizer access to the data structure inside the holder, which is the syntax graph

-] have the visualizer build its own data structure?  Or augment the existing one with new visual related stuff?  Add a visualizer field to all of the syntax graph nodes?  Make it an array, one spot for each visualizer added to the source holder?

-] Go with augment for now, even though feels like it's mixing things dangerously..  it, on the other hand, puts the visualization right into the data visualized, so keeps the two in sync -- no danger of something modifying the data behind the back of the visualizer..

-] make the augmentation for the one syntax graph element created above.

-] have the visualizer generate a visual element object from that simple graph.

-] have the visualizer send that visual element to the display created above

-] The javascript structure of linked objects will be internal to the source holder.  The visualizer will take that and construct stream of visual elements from it.  The modifier will directly modify it.  The holder will also have a serialized version of the data structure -- which is JSON format.  The data structure will be made such that can use the JSON tools to directly turn it into JSON, save that to a file, then later read the file back, and use the JSON tool to parse the JSON back into the javascript linked objects.

-] The modifier will attach additional objects and properties, which are used to manage interaction with something that is editing the source..  for example, an insertion point is maintained, and highlight markings, and selection markings.  (maybe multiple selections, and even browse through previous selections, and fork off one)

-] The modifier will notify the visualizer, somehow, of what changed, each time it does a change.  The visualizer can then decide what needs to be sent to the display, and can decide new visual placements of things, if needed.. (thinking maybe have delimiters in the graph, that mark the boundaries of what is being visualized, and act as anchor points for pulling in things from off-screen..)

//=================  From graph to placement and size  ==================
//
//=======================================================================
Given a syntax graph, need an algorithm for deciding what are the local contexts, building up the hierarchy, and positioning each element inside a given context, and scaling that context relative to its parent (all siblings should have the same scale).

Okay..  first, have connections between different graph elements..  want connected things to be visually close..  second, want to include user-specified hierarchy, so entire sub-graphs can be tagged as one "unit" -- the equivalent of a function -- such units can be re-used in multiple places..

Will need different view annotations for each place the function is used..  so, maybe add typed properties to the original, where each type says which reference it pertains to..  so each reference can have different view arrangements..  Or, maybe, 

//==

# Okay, what about this..  place annotations, by hand, onto the syntax graph objects..  those state placement relative to each other, and state visual hierarchy, and state scale relative to each other..  then, the current view is like a projection onto the graph, that selects which elements are within the view, and sets a global zoom.
[-] Done and working

# You know, pretty much want to build the view hierarchy as part of the syntax graph..  will need visual hierarchy, including bounding boxes, in order to calculate what is (potentially) visible to the camera, and when go to 3D, will need hierarchy in order to know things like what to represent as a single box, versus to break open and show the innards..  and hierarchy will decide relative scale to each other -- siblings at same scale as each other, and parents at larger scale than children, and once scaled small enough, stop showing children, which are the innards, instead just show the parent as a box.  Also will want hierarchy during projection from 3D onto the camera's view..  hierarchy will affect placement within 3rd dimension..
[-] Done and working

******           ***********     ***********    ********
*Will, at some point, want topological visualization, which represents translations between worlds..  represents the twist-upon-itself topology of many types of thought..  the thing where, when think about some system, there are relationships among things such that part of thought moves within one "system" or one "universe" and then a transition happens to a different system/universe after which have relationships within that one, then switch again..
******           ***********     ***********    ********

=========================================================
== Attach view hierarchy to syntax graph
==========================================================

#-] attach the view hierarchy to the syntax graph itself
#- -] The Display receives pointer to the hierarchy, but it is embedded within the syntax graph, as an extra field that exists in every node, but the field is not visualized.

XXX -] Be able to generate view hierarchy code from just data, calculated.

#-] Make Persister and Restorer, which write the syntax graph out to file as JSON and read it back in.  (see design notes for algorithm)

======================
= Event Handlers
======================
# -] attach a handler fn to each bounding box
# - -] test that the smallest bounding box is what receives the event 
# - -] place parent links into each bounding box in the view hierarchy
XXX == Try invisible box over the top of text
#== try handler fn loaded onto the html page and spec handler name in the svg
== search for others experiences with famous event handling with svg
#== post a bug to famous..?
#== try writing the svg directly to DOM, see if events work in that case (ie, rather than that famous is messing it up)

#-] make the Display register a Famous event on each surface that corresponds to a bounding box
#- -] make the Display put a back-link in each surface that points to the view hierarchy bounding box that the surface corresponds to
#- -] it receives the event object
#- -] it takes the surface out of the event object
#- -] it uses the back-link in the surface to get the view hierarchy object

#-] make a handler for key presses.
#- -] the handler receives the event object, 
XXX - -] plus the smallest enclosing bounding box.

# == try keydown on whole document, plus mouse over, mouse exit, to track which surfaces "receive" the key

XXX -] Get it so that keydown is detected on individual surfaces, and the event object holds the particular surface..  as well as x and y relative to window

# -] See if can attach an event handler directly in an SVG string, which names the generic Display handler.
[-] Can attach mouseover and mouseout hdlrs, but not a keydown handler
#- -] set the innerHTML of a DOM element to be the SVG string.
#- -] put the generic Display handler as a script in the HTML of the seed page

[-] keydown and keypress don't work on most innerHTML, and don't work on Famous surfaces that have SVG text inside them.

-] make a generic-event-handler, which is part of the Display code
- -] it calls the handler function that is attached to the view hierarchy object..  hence, handling function is specific to particular view box instance!
- -] when register an event handler on a famous surface, register the generic Display event-handler
- -] make the holder supplied event handler internally dispatch on the type of event!  hence, all view boxes have zero or one event handler.
- -] The generic handler only gets the famous surface (or DOM element) out of the event object, then gets the view box from that, then invokes the one-and-only handler that's in the view box, giving it the event object, upon which the handler dispatches and does view-box specific behavior according to event type.

-] Try approach of directly populating the DOM
#- -] see what happens when attach object.style.transfrom="trans-fn"
XXX- -] use object.style.transfrom="trans-fn" for shift and scale transforms
[=] Erratic..  not reliable..

[=] SVG directly inserted into DOM works!  After much experimentation
-] Populate DOM with SVG 
- -] put one root SVG in the HTML page
- -] create new SVG elements, add SVG string as their innerHTML, add as children of the root SVG
[=] adding shift transform to SVG elem doesn't work, but setting it's x and y does!
- -] accumulate offsets from ancestors, to arrive at exact x and y of each SVG shape, relative to the root SVG's origin.  Also accumulate the scalings.
#- - -] take a few minutes to look inside Famous code to see what they do
[=] got nothing from the code..
- - -] keep a chain of all ancestor transforms -- each view-set-link to a parent, going up to the root, has a shift + scale transform on it.  That represents the shift applied to the root container and the scale applied, in order to place that entire view set at the x and y within the root svg, and arrive at its scale (which is applied to the view set's container's size).  Keep the product of those transforms as travel around the view-set graph.  For each view-set, have one composite transform that applies to all elements of that view set! 
- - -] As render a given viewset, make a container for each view box that has children, and make an SVG element for each rendered shape.  Calculate the composite transform, given that of the container and the offset and scale of the child within that container.  Each viewbox has its own shift and scale transform, which is relative to its parent viewbox (or view set container, for the root viewbox).  
- -] Insert a back pointer into each DOM element that points to its view box
- -] make generic event handler obtain the DOM element from the event object, and from there obtain the pointer back to the view box, and from there invoke the event handler that was attached to the view box by the visualizer, and has behavior specific to the view-box.

-] Try approach: 
- -] put mouseenter and mouseexit (mouseover and mouseout) handlers on all the famous surfaces.  
- -] Track order of triggering -- mouseover adds to end of list, mouseout removes from list.  Only issue is when have a ring around another surface.  
- -] Only use shift and scale modifiers (to reduce chance of Famous transform bug).  
- -] Separately add a keydown handler to the DOM document as a whole.  When keydown detected, then key is inside of all in list.  Directly inside end of list.  Keep a head pointer and a tail pointer.  Doubly

-] make menu pop up in response to keydown
- -] do canned menu contents, for now.
- -] make menu contents include keys to press and include a handler that dispatches on the particular key.
- - -] Some mechanism in the handler will have to recognize when a click is performed instead of a key press, and performed outside the menu boundary, and so cause the menu to go away.

-] when particular kind of event happens, calculate the mid point in the side of the nearest shape of the appropriate type.  For example, when adding an arrow, hit a particular key when the mouse is near to the side of a drawn port node.  First find that near port node, then second calculate the mid point of its nearest side, third create an SVG bezier element from the side mid point to the mouse position.

-] make the gesture-handler turn the event plus bounding box into a command.


==============================================================
=
=   Use cases
=
==============================================================

-] hit space bar, which brings up the menu that shows the key mappings.. 

-] make code that draws the "menu" rectangle
- -] give it a z that is in front of the other surfaces, perhaps 80% opaque

-] hit 'e' key near an elem box, have an arrow spring out, from "portsOut" to the cursor.

-] move cursor, have the arrow track the cursor

-] hit 'e' key again near a different elem box, have the arrow terminate at "portsIn" of that elem box

-] over empty background, hit 'a' key, have that pop up a new elem box

-] near that elem box, hit 's', have that create a new properties box and connect an arrow from the elem box's "properties" over to the new properties box.  (If can't find a nearby elem or other properties box, then just make an orphan properties box?)

-] hit 'f' key, have it zoom in on the box under the cursor

-] inside the box, hit arrow keys to move cursor between fields
- -] at each field, use space to pop up another menu and then space again to un-pop it..  
- -] make this menu relate to the valid entries for the field.  For one field, show valid sequences that uniquely identify one of the valid values for the field (ex, for the "element type" field).
- -] make menu be hierarchical -- first menu chooses a field via a key -- second menu shows all unique sequences for values in the field
- -] have the menu for the "variable Name" field simply say "type in alpha-numeric text, which can include spaces but no other whitespace)

========================================================