Architecture Requirements

The following analysis is performed from a position of practicality and (surprise!) authoring for novice-intermediate users. Consider this an author-centric design, akin to player-centric or user-centric design philosophies of game and HCI design.

By “practicality,” I mean attempting to maximize the interest of potential authors both inside and outside academia.  To increase the number of potential authors (one of my primary goals), I am choosing to include non-academics wanting to create dramatic interactive characters in my target audience.  This means that I cannot assume my users will swallow a system if it sacrifices too much usability or common-sense logic in favor of a particular academic theory.  For example, Haskell is a beautifully constructed language that is worth learning and studying, but it is rarely used in the job market instead of Java, C++, or another functional language.  Phrased another way, the more of the system that can be explained in terms of common-sense or folk psychology (rather than scientific psychology), the better.

So, while “practicality” has expanded my target audience tremendously, I must attend to those new (novice) potential authors. I can assume they have some programming knowledge — a BS in computer science, self-taught a language or two, or fundamentals for an entry-level position in CS — and have familiarity with logic and the flow of code.  The price of admission into using whatever architecture I choose must be as low as possible to ease these authors into its use, and I fully expect to have to reduce its price of admission beyond what it is currently.

However, I also want the architecture to be able to satisfy the promise on the box: to be capable of making realistic, dramatically interactive characters. I do not want to make a training program with training wheels that cannot be removed, a program that is discarded once users reach intermediate level because it is too simple.  The author should be able to create whatever behavior they can reasonably imagine, given enough experience with the system. The system should also not break down as an author reaches a satisfying complexity for their agent, in logical horsepower or efficiency.  There are, of course, reasonable upper limits: we cannot expect the user’s agent to solve AI-hard or AI-complete problems.  However, if the user did their best and spent hours coding the door-opening behavior… and then the performance lagged so badly when deciding how to open the door that the experience was unplayable… That would be a catastrophic failure.

From an authoring-for-authors perspective (mine, not the user’s), the behaviors should be hierarchical and capable of modularization. For example, once the user creates the fully-fledged behavior to open the door, they can forget about it and trust that it will work.  An even lower-level example would be head-tracking: telling the agent to “LOOK HERE” and trust that their head won’t spin around Exorcist-style and seem to break their spine, but they will instead move their body appropriately.  This enables authors to create libraries of behaviors that can be shared, reducing future authoring burden.  When authoring a higher-level view, such as a scene, the less the author doesn’t need to worry about the low-level details, the better.

In summary:

  • Favors usability and common-sense logic
  • Low price of admission
  • Capable of higher complexity
  • Scalable
  • Hierarchical and Modular

I have been, up until now, using a few choice buzzwords to describe the type of agent I want to support: virtual (a given), dramatic, interactive, embodied.  “Embodied virtual characters” necessitates that the agent not just be a mind in a theoretical body making decisions, but instead that there is a physical form (in virtual space) being driven by the mind.  While I have punted dealing with implementation details of the agent’s inputs/outputs, the agent should still have functional signals and embodied movements. If the architecture is not capable of handling inputs/outputs of an embodied character, the author (or I) would have to make or support them.  Interactive implies that not only can the agents created interact with one another, but with a human player/user. The interface to that human can be anything (mouse, keyboard, kinect, brain waves) so long as there are no crazy hoops to jump through in order to include a human in the play loop.  Finally, I describe what I mean by dramatic agents, the gooiest word of the bunch.  To quote

adjective \drə-ˈma-tik\:
sudden and extreme; greatly affecting people’s emotions; attracting attention : causing people to carefully listen, look, etc.

On first glance, this word sounds more like an authoring challenge rather than an architecture specification.  Indeed, I will be tackling how to support authoring dramatic characters shortly.  But authors, especially non-academic ones, do not want to make boring/lifeless characters in most cases.  It should be as easy as possible to make punchy, sensational, and exciting characters.  The less work that I have to do to make the architecture support dramatic realizations of the embodied characters, the more attractive that architecture is for me to use.

What exactly does “supporting dramatic realizations of embodied characters” mean?  I need to do more research in this area, for starters. But for now, I attempt to describe it as… providing architecture that treats different personalities, extreme emotions, and the general expressiveness of the agent as important concepts.  It goes back to folk psychology and the hierarchical concept I hinted at above: authors know that an angry person might enact “opening a door” differently from a sad one regardless of other logic involved.  It has to be possible (and hopefully intuitive) for the author to change the expressiveness of the performance (which is related output specifically, but is still part of the agent’s AI), while not tangling it up with the functional decision-making process.

The architecture should support creating agents with the following properties as much as possible:

  • Embodied
  • Interactive
  • Dramatic
  • Expressive

Now that we have our requirements (with a touch of gooey nice-to-have’s), we can examine a variety of architectures that concern themselves with decision-making AI and choose the best candidate.

Posted in Uncategorized | Leave a comment

Tackling Authoring

While purposefully avoiding framing the problem in terms of a specific architecture, calling upon more sophisticated data structures, and locking down a specific output method (which was why the ‘coding’ and ‘performance visualization’ sections were so slim), I have presented the authoring dilemma through a working example.

Why authoring? Regardless of the implementation details of the artifact or architecture, someone has to make content (in this case, I focus on the AI) that fits into whatever restrictions are required.  Even grammars and procedural generation need some terminal assets to work with.  You can’t make something from nothing. And if we want more behaviors, characters, scenarios, or whatever we are aiming to build, we need some combination of more authors, more experienced authors, and ease of authoring.  I feel like phrasing this in terms of an equation would be more clear (and someone else has probably done this better than I have already). Something like…

Content Authored = (#Authors * Avg. Skill Level )/Task Complexity

(Note: I am ignoring time as a variable in this discussion because it’s a constant that we cannot affect. Its effect can be observed in #Authors, as I am implying that more authors in the same span of time can produce more content. Also, # of authors or increase of skill does not scale as this equation implies — it is all case/task-specific. This equation represents the general positive/negative relationship of the variables involved)

On a specific project, teams manipulate these variables as best they can to maximize quality output: recruiting more authors (+ #Authors), training their authors (+ Avg. Skill Level), and utilizing hierarchical structures/building in-house authoring tools (- Task Complexity). This is well and good within teams, but my personal goal is to make it a reality in a broader sense.  I want independent game designers, game companies, and more people in general to make use of the wealth of research that academic institutions are creating.  As I said above, no matter the context, academics and non-academics alike must be able to create content in order to create these interactive virtual characters. It is my theory that the authoring burden is too high for anyone but the most determined, technically trained and scientifically-minded to want to use these systems.

Authoring Burden = SetupCost + (#Tasks * Avg. Task Complexity)

This equation is more common-sense than mathematical. It is clear that if someone has to make less things (- #Tasks), and/or those things that have to be made are simpler/smaller (- Avg. Task Complexity) , then it is easier to make those things (- Authoring Burden).  My theory is if we reduce the authoring burden, more authors will want to use the system. As a community, we grow; we get more interactive characters enriched by a wider pool of perspectives.

If we build it, they will come. Great. How do we build it?  There are many great minds that have implemented a number of architectures that make interactive characters not only theoretically possible, but physically possible. I do not intend to re-create the wheel. My goal is to choose an existing architecture that is optimal from an author’s perspective and make it more so by extending it to alleviate its authoring burden.

I have outlined here what makes up the authoring burden in general, but before I can assess how to alleviate a specific architecture’s burden, I must pick my architecture. The above writing will help me ground further discussion in how to do just that, which will be the topic of my next post.

~~~ ***** ~~~

(Note: Paulo say that arguing for “Complexity” in an academic document isn’t a good idea. Why? I dunno. We can find a better rephrasing of it to be measured quantitatively and qualitatively, such as: Number of clicks to do a task, Number of lines of code written, Number of functions, concepts, or interlocking sub-systems involved, Time to train authors to accomplish the task, Number of bugs/possible mistakes…)

Posted in Uncategorized | Leave a comment

Working on Advancement

The paper was finished, submitted, and accepted as a full paper at FDG. WOO! I need to do revisions in a couple of days.

But first, I’m working on my advancement introduction! I’ve been struggling with it for the past month or so, getting information, changing details and framing, and overall pummeling my ideas into the dirt. I’ve gotten to a pretty good place so far!
~~~ ***** ~~~
In my paper, I touched on why creating interactive virtual characters are difficult: they have three huge sections (sensors/input, the AI that acts on those sensors, and the actuators/output that enact the AI’s decisions). This is fundamental in any standard AI textbook and is true for any autonomous agent.  Understanding real-life inputs (through something like the kinect) is a hard problem on its own, as is procedural animation of arbitrary actions. The focus of my research is not on those areas, but on the gooey center: the AI that determines behavior.

Even ignoring those complicating parts of the process, authoring behavior is very difficult on its own. Working with an example quickly illustrates the difficulties of authoring.  The following is one instance of brainstorming how someone might conceptualize the behavior of opening a door.

  1. Assume we start with a single idea representing our idea of “opening a door”
  2. Well, obviously if the door is already open, this behavior should do nothing. So the character should only try to do this behavior if the door is closed!
  3. What if the door is closed, but is locked or blocked in some way? We need to execute a sub-behavior to unlock or unblock the door, which would start going back up to #1 again for a different behavior. The same would have to occur if you are out of range of the door.
  4. If you get distracted from opening the door, such as a kitty walking by demanding attention or an earthquake shaking the building and destroying the door, when is it appropriate to abort or continue the performance?
  5. And what if those sub-behaviors fail, if the door is nailed shut or ‘blocked’ by a sign that says “KEEP OUT!” (physical and social barriers)? Should you keep trying at the door? When should the character give up trying to open the door?
  6. Let’s say the door has a non-physical property that would cause it to be treated differently: such as a locked bathroom door, or a front door to a house that isn’t yours. Now we need to deal with complicated social norms of knocking, waiting, and maybe even conversing with people before we can open the door.
  7. What if you are in danger, and you see an armed person on the other side of the door? You certainly wouldn’t want to try to open the door any more! ABORT!
  8. Or, better yet, what if you have the door half-way open before a rabid dog comes barreling down toward the door? You not only want to abort, but reverse, the performance.
  9. And if you finally get to be able to open the door, certainly not everyone opens the door the same way all the time. If you’re angry or stressed, you’ll swing that door open quickly and tersely. A secretive person, if they suspect someone is on the other side, will crack the door open and guard it. A butler would open the door wide and stand at gracious attention.

What started out as a single idea now contains a huge chain of sub-behaviors gating the performance of opening the door, including possibly deserting the behavior all together, and an overloaded performance of how many different actors could dramatically open the door. And each sub-behavior expands exponentially as this single door behavior has done.

This all assumes you also know how to make all of these behavior cases execute properly in code: such has animating door-opening speed, or keeping track of social norms like bathroom use. And, even if they did execute with flawless logic, that the player/viewer would read these performances as they were intended: what if a fast door-opening instead read as excited rather than angry? The author may have to iterate to make sure facial expressions and other supporting evidence in the scene communicates their proper intention.

My research aims to address and alleviate these authoring problems (of rapidly expanding state space, code-writing, and performance visualization) so that creating detailed, interactive, and dramatic stories is not such a daunting task.

More to come tomorrow.

Posted in Uncategorized | Leave a comment

Final Wave of Interviews

I met with a ton of people last week, including my adviser, and have made a lot of progress in bringing the paper together. This post will summarize the paper-relevant feedback I got from the meetings/interviews before I jump into taking a stab at a full first draft.

Michael Mateas
While I have been able to meet with three groups and get three different graphs of their authoring processes, they (as I mentioned in my last post) didn’t converge into a similar-enough shape. In many ways comparing the three systems (ABL, FAtiMA, and BOD/POSH) was comparing apples to oranges. Being the most complicated and in some ways encompassing the other systems entirely (in terms of their general purposes and components), we decided to focus on ABL in the paper and reserve the other systems as supporting evidence to our claims. PAPER NOTE: Going to need to describe how they are different and why we chose ABL in citeable ways.
I identified what I still temporarily call the “magic step” in authoring in all of these systems, which can be defined as the process by which an intermediate/expert author processes an abstract task and creates the functional code that represents this task. “Abstract task” here is the specs of the behavior in natural language, and the “functional code” output is the code that can successfully be run by the system and drives the agents in the system do perform the abstract task according to its specs.

All systems share this magic step, but the process within the magic step is different for each system. A “useful” authoring tool for a system must quantifiably alleviate some part of the magic step, transferring the process from a human author to an external source.  This is my personal definition for what I call “useful”. Also note that the eternal source does not have to be digital or computational: a white board where you offload a process as a process map is technically an authoring tool.  However, if the whiteboard doesn’t save time (is this our ultimate definition of “quantifiability?”) or help us make behaviors that would otherwise be too complex for someone to hold inside their brain, it is not “useful”? Is this definition going to be troublesome? Maybe!

Behavior-Oriented Design
Last post detailed the magic step of BOD/POSH because of its simplicity.
I had some words here, but they went away. I’m sure Swen will be able to articulate what I’d say better anyhow!

2013-11-22 13.42.25
I spoke with Paulo in order to tease out the magic step of FAtiMA. While I thought it might start with a list of abstract actions like BOD or ABL, FAtiMA is primarily concerned with goals and motivations instead. Teasing those out in quantifiable portions is the key challenge of FAtiMA authoring.
The whiteboard up there shows the process. Instead of starting with actions, you start with goals (being altruistic, personal safety/fear) which are driven by the decision points (Helping the soldier find his lost interpreter or not) which are often broken down into sub-decisions (Analyze the photo or not, tell the soldier the info or not). Each decision must also have a quantifiable reason to pick one alternative or another (Wanting to help vs. 10% chance of being harmed).  Actions must also be made to support all the agents moving through the three (moving/speech acts that are implemented separately).  Finally, you make sure that different people have different personality scalar values, which ultimately help choose which decision path each person takes. It takes a lot of trial-and-error testing to make sure the decision paths work as intended! [Samuel at Portugal said his most difficult authoring tasks were the tedious number-tweaking].
Paulo shared an interesting insight about authoring his recent paper.  He spent a while building up his chain of goals and decisions, but he was having problems with simultaneous goal consideration. Paulo had to contact a more experienced FAtiMA author to learn an implicit limitation of a system: that an agent can only consider 1 active goal at a time. That means that any simultaneous considerations must be made as intent goals.  We want the authoring tool to inform the user about these necessities to save valuable time, as Paulo had to redesign his structure based on the system’s limitation. The act of writing an authoring tool forces questions like these to get answered (Can there be more than 1 of X? What do you need to complete a Y?)

Speaking of referencing more knowledgeable people…
Every ABL author has to ask for help or reference at some point or another, but novice and intermediate authors have to do it A LOT. ABL is its own language with no IDE or language support other than a text editor and filterable print statements.  Debugging it is a nightmare, and it has huge room for improvement.  The following interviewees are listed in increasing order of ABL authoring experience.

Will works on connecting the new assets gotten from the animators and other basic functionality (like locomotion) and connects them as ABL acts for programmers to use. The most authoring he’s done in ABL is using behavior template things for super basic stuff that involves only one character (that I’m not familiar with because I never had to touch them). Will wishes for an IDE and basic language functions (like iterations, If-then, etc) to help reduce code bloat. He imagines a kind of isomorphic debugger that will help debug logical errors as well, but something more usable than the ABL Debugger because he thought it was very intimidating.

2013-11-26 12.30.29
The above image outlines Karen’s behavior authoring process. In ABL, the behavior being authored is called an SIU: Social Interaction Unit. The first step involves sitting down and, in great detail, figuring out what the behavior should do. The process involves understanding what signals, WMEs, Animations, and Responses will be used/required, and if any of them are missing Karen must make sure they get made. Andrew will also point her in the direction of similar code that has been made in the past for her to use as examples. This whole first process is the most crucial step, because if Karen asks the right questions and it is clear what is expected of the behavior, there will be less tweaks to author later. Karen also notes that she rarely is asked to simply author a whole behavior, as it is the tweaking/adding functionality bits at the end that require the most of her attention/task allotment.
Coding then begins. If anything is needed from other people, she contacts them. If she needs to use other people’s code as examples, she asks them to explain their logic.  If she runs into trouble in debugging, she will ask Larry/Josh/Andrew in that order (going up the food chain).  She takes notes along the way if there are any decisions she had to make that she did not forsee (ex: Andrew may not have told her HOW to make something gracefully halt, so she tries something that she thinks will work.) When the list of things she distilled from the initial spec is completed the “First Draft” is done and a meeting with Andrew to discuss it is scheduled.
The second big meeting is where everything is analyzed. Always, specs need to be changed, animations need to be tweaked or replaced (which may require outside waiting), or the entire behavior of the SIU needs to be changed (ex: “watch others” was meant to be a short enhanced idle, and turned into the “reactionary state” in which the Head of Household is always in, ready to react to disturbances). Further iterations on the behavior authoring and Andrew-meeting occur. And once the behavior is pronounced ‘done’ or at least ‘done enough,’ Karen may be asked to tweak/change its animations, performance, or interaction with other SIUs at any point in the future (what is considered “polishing”). An SIU is NEVER DONE.

Karen’s interview offered great insight into the interpersonal process by which ABL SIUs are authored, but not so much into her personal coding problems. She feels that she is novice enough in the project that her main operations are copy-pasting existing code and tweaking it as necessary. An authoring tool would be able to offer her less code support (other than extending currently-existing templates) and more performance-based visualization.

2013-11-25 12.14.38Larry’s interview was exceptionally fruitful, as we dug deeply into his current authoring task: aiming a rifle. This act is extremely punchy because it has very extreme social repercussions, and it is very complicated because it must function between the player and NPCs, as well as for the case that the Initiator and Responder both have rifles. Because of the extreme social repercussions, handling the reaction to a gun being raised/lowered is very important — what distractions, and for how long, are allowed to interrupt the performance of the SIU? Very muddy and difficult questions for ABL authoring.
What you see in the image above is 3-4 interconnected sub-trees of ABL logic showing the raise-rifle SIU performance of the PC/NPC, Initiator/Responder performance, possible recursion, performance conclusion, and interruption resolution. It would take a long time to go into detail about each of these graph partitions, probably more detail than we have to spare in the paper other than with a complicated-looking diagram. What is most important about this, however, is the need for a graph visualizer for the flow in ABL, something more usable than the ABL debugger. Larry needs to be able to show, easily, that the different roles using this SIU will behave properly going down different appropriate paths, including with other SIUs interrupt. A template should also be able to be generated/created to use as an example for such complicated behavior in the future. And not just tree structures should be able to be visualized, but the repeated performances by all the agents should be traceable and easily seen as well.

Josh has not been part of the behavior-authoring gig in a long time, but he was the author of some of the original SIU structures and basic templates that exist now. He primarily works on the ABL compiler making language improvements, specifically adding CiF social rules to the ABL structure. (Making it MORE complicated to author, AAAAH!!!) But he was the most helpful in showing me how to run the ABL Debugger, how (in general) to make an Eclipse patch for basic IDE help (which Claudio Pedica supported), and other ideas for connecting tools to the ABL runtime code. I have many clear authoring paths forward (for future work in ABL).

IMMERSE Project Coordinator and leader of our little band of coding misfits. He helps design the scenarios and the SIUS that are needed for the scenario behaviors. He codes infrastructure and SIU support so that all the other authors know where their SIUs sit in the grand scheme of things, and also how their SIUs may be interrupted.  His authoring methods were described as follows:

  1. Figure out the spec in descriptive terms: Initiator/Responder, Signals, and the logic of decisions
  2. Make a basic version in the code of step 1
  3. Look at the basic version from step 2, improv/tweek it, and make sure it plays nice (interruptions) with other SIUs

This process supports Karen’s description.
Andrew agrees that tools wouldn’t be as useful for him because he is so experienced with the code base. However, it is clear that help with visualization and debugging is needed. Currently all they have now are Josh’s code templates. A better animation viewer would also be nice. Josh’s task to make a way to test SIU sequencing without animation was in the works, but never got finished.
The biggest coding road block are the visuals: turn around time is slow, recording to get them made is slow. Nico is our only interface with the animators, and the animators don’t ever get to see how their animations play/fit in with the bigger system. There are also technical bugs with the animation system, animation blending, and the performance manager (which doesn’t have animation blending yet).

This section pretty much sums up what I spoke to Josh about. Listed in rough order of difficulty and time, at least by my estimate:

  • Eclipse Patch: Requires so little domain knowledge it’d probably be more suited to one of Lyn’s group because it requires making a grammar. Josh estimated a day or two of someone sitting down and just making it. Would help with Eclipse coding support that some people (mainly Will and more industry folk) would like.
  • Animation-less Sequencer: A different IMMERSE build (but would work with any ABL runtime) that would skip/ignore animation code and only worry about action sequences. Should have a gui about what actions to keep track of, and maybe some other intelligent filtering. Maybe specifying what sequence you’re expecting and catching derivations? Separating out animation from logic debugging is a key advantage that FAtiMA/BOD/POSH/Basically all other systems of this type have
  • ABL Debugger 2.0: Revives the old ABL debugger. Adds a more searchable/configurable behavior tree visualization (seeing multiple zoomed sections of the tree at once. Saving those zoom points for fast iteration). Easier/better break points. Maybe stepping behaviors forward? Possible re-runnable GUI for quick iterations on small parts of the tree. Maybe a search function to explore all possible instances of code written? Including interruptions and signal noise. Basically allowing for robuster code.
  • Better Animation Viewer: Possibly linked into the ABL Debugger. Most likely linked into the ABL runtime. Expanding the ABL debugger with graphical stuff, basically. Might need to brainstorm what this would look like/how it would be different from the ABL debugger…. or how much more it would take.
  • Some support for Social Games — not really implemented or well-known, so no idea how hard it will be to author or debug.

I can see some Javascript D3 visualization for any of these but the patch one. Which we can connect via local signals to a local webpage.

Quick Aside
I did interview Claudio Pedica, lead designer and programmer of the Impulsion Project. It was written in C# using Unity’s behavior tree library with some custom tweaks. He agrees that we need a tool to visualize the state of a BT because what makes up the state is spread all throughout the tree. And that it is very hard to debug between an error in the code and a character that is misbehaving. Reactive planning is great at helping make sure characters respond to the right thing at the right time, but it makes debugging the ‘behavior sequence’ extremely challenging.

We have 3 cases where we analyzed a behavior (lost interpreter) and the authoring process to create it down to the code level. Each case shows a different level of complexity, using a system with different philosophies behind them, but all of them share the “magic step” by necessity. This paper analyzes the “magic step” process of each of these systems to better understand what would be required in an authoring tool to alleviate the author’s burden in executing this step. Hopefully the details made explicit here will aid any group looking to reduce the authoring burden in creating interactive characters.

Posted in Uncategorized | Leave a comment

Worry Worry…

Imposter Syndrome is taking hold as the deadline approaches. December 6th is judgement day.

I have interviewed a decent number of people now, and the graphs have not converged as expected. Half of the people I spoke to didn’t have graphs to give me. The following is a summarization of what I learned from each interviewed person and how it’s shaped the overall paper.  My goal by the end of this post, or at least in the very near future, is to find some commonality between all those I’ve talked to which can be ground out in a concrete suggestion for improvement. That suggestion must then be tested by different teams and results given.

Larry (informally) and Karen: ABL
Some of this was apparent in the last post or two. I got a graph and a discussion from Karen that detailed the scenario-authoring pipeline where the largest and clearest time sinks came from the graphics side of authoring. Having all assets, it would take roughly a day of tweaking animation and code to get a specific low-level behavior, such as picking up an object on a table, to work and look well. Larry’s feedback was more on the ABL-level authoring: during that day of time, there would be lots of weird debugging tactics that would go into sucking up attention. Besides the process from writing code to seeing characters performing the action is so long, there is lots of room for errors to halt the process. Lastly, as Karen put it, there needs to be interfacing with higher-ups to make sure the behavior performs up to spec. It takes a lot of back-and-forth to figure out just what the behavior should do, how it should look, and (once it is made to do that) was it really what we wanted in the first place?
2013-11-05 13.54.07
In summary, improvements can be made on all levels of the authoring process identified through delays between other team members and in performing individual tasks.

Martin Van Velsen: Authoring By Demonstration
Martin introduced me to a field of authoring I was unfamiliar with: authoring tutors. Human A authors a tutor to teach Human B some subject matter, be it math or some other easily verifiable and highly computational task. He spoke of what he’s working on now: authoring by doing! Human A sits at a tool and solves a math problem while the program tracks these “correct” paths to solving the problem. Human A then explores alternative paths of easy misunderstandings, such as adding fractions by adding numerators and denominators at the same time (1/2 + 1/2 = 2/4). The tool is gathering data of correct and incorrect methods, and maybe would then be able to extrapolate the methods to other fractions or data patterns.
An interesting approach that would require more thought to apply to our domain. However, his remarks on tools and visualizations therein were very useful. For example, accepting that very few (probably no one, other than you) will become experts at your system. (I see this as Andrew being the expert at ABL, and everyone else putters around doing the best they can). Deep visualizations that show a lot, or all, of some data will be indecipherable and thus useless to most users. “Wizards” to help author certain things over and over get repetitive and insulting: once people are trained to do something, they should be able to do that something with a minimal amount of clicks.
An aside: tools like Narratoria were similar to ours, but they authored for cut scenes instead of richly interactive experiences. As such, their script + movie + brief dialogue-tree-like interaction authoring tools were sufficient.

Samuel and Paulo: FAtiMa
FAtiMA’s strengths lie primarily in its focus on emotion, grounding in research, modularity, and its complete isolation from the enactment side of the experience. I wasn’t able to extract a specific graph from Samuel, but I did get references to other projects and an overview of some of the bigger projects that FAtiMA has been used in.  Most recently, the focus of his research has involved a new module about “Social Importance” and makes FAtiMA’s goals feel less utilitarian. Every action considers the social context in which it exists.
One of the more amusing observations is that FAtiMA’s strengths are also its weaknesses.  It is difficult to conceptualize a model that doesn’t include emotional responses (see Paulo’s thing later), so it feels like you’re strong-arming the system into doing something it wasn’t meant to do. Some of the research that FAtiMA is grounded in feels inflexible (the plans that Samuel feel do not always capture the decision-making process well if you do not have a concrete end state). And being isolated from the enactment means that physical properties related to behaviors outside of FAtiMA are hard to handle (ex: FAtiMA has no model of eye gaze).  The biggest authoring challenge by far, however, is number tweaking. For example, assume every action that an agent can take has two forms: formal and informal ways of expressing the action. “Number tweaking” refers to testing and discovering the threshold of some numeric variable that determines at which point each agent does one or another.
Paulo and I walked through two FAtiMA scenarios that reinforced the strengths and weaknesses talked about above. ABL would likely become easier to author for if we had libraries of pre-programmed actions that the user grounded out in performance. Likewise, FAtiMA would likely be much easier to use with loads of pre-calibrated modules representing different social considerations and theories (OCC for emotions, social considerations, eye gaze, etc). But those are pie-in-the-sky ideas.
2013-11-19 11.04.582013-11-19 13.52.22

Swen: BOD / POSH
Simplicity is key. Swen and a different Czech team have been using that as their motto to make a decision-making system. Swen and his adviser have a simple multi-step process Behavior-Oriented Design (that I’ll describe in detail shortly) between the programmer and designer to craft the tree. Their tree structure is very controlled, and they have a visual editor for designers to use which helps in making the structure.
BOD involves the following steps: Designer (D) + Programmer (P) work out every action they want the agent to perform in an abstract way. D + P then work together to distill those abstract ideas into Actions and Sensors, what will ultimately become function names. The P then creates the list of functions in code, and D then uses an editor to make the behavior tree. Any hiccoughs that occur in the process (ex: forgetting an action or sensor) have the D + P go back and make revisions as necessary in an iterative process. If any of the behaviors were left as stubs or disconnected from the final system, the P will fill out the functions as necessary once the graph is complete.
2013-11-19 14.54.59
The work pipeline is smooth, explicit, flexible, beautiful…! It’s as complicated as it needs to be for the system. But how can we abstract the process for more complicated systems? Is it even possible?

Ugh, took way too long to write this and to share my notes. Sorry! Follow-up posts to come shortly.

Posted in Uncategorized | Leave a comment

Paper Outline (Mark Nelson is Awesome)

Catching up on two weeks of work without updates: GO!

Besides reading a bunch of papers, I’ve begun the info-gathering process of the paper from authors of ABL. I need more authors of different systems to say anything broad about the interactive character authoring part of this, but it’s a start.

Mark Nelson and Michael Mateas’ Requirements Analysis for Videogame Design Support Tools has been immensely helpful for structuring my goals for the paper as it is very similar to what I’m trying to do. What they were reasoning about was formal, so they had a bit more concrete subject matter, but the fuzziness of people using and understanding their tools is very similar to mine.

Abstract: will be written last!

Introduction: A description of the interactive character and its uses (in games and drama) and the rough problem statement that they are very difficult to author. A combination of mental state and reasoning, procedural animation, and (in most cases) natural language processing: making a human-like character is insanely difficult and requires a LOT of content to be contributed from vastly different disciplines into one cohesive project.
Different groups from different backgrounds have attempted parts or even all of this feat, but each project is cumbersome, challenging to author in, and ultimately falls short of this task. The following analysis will hope to clarify the major challenges in authoring interactive characters and offer insights that may be applied across any similar authoring system.

Design-Support Systems: Nelson analyzes CAD as a starting point/an analogy for a semi-formal design-support system that integrated computers with the design process.  He speaks of systems that supported the vocabularies of different disciplines in order to aid more narrow-focused designers (for me, it would be a look at Narratoria). It is useful to note here that early in the design process, many disciplines use sketches, rough drafts, quick ideas that get zipped down easily without worrying about complication (find source!). A “mode” of that would be a good idea. Finally, the idea of “enhancing creativity” is mentioned whose words can pretty much be applied word-for-word to my problem as well.

Schon proposes four main uses of a design system: enhance the seeing-drawing-seeing loop, allow construction and exploration of microwolds, help manage a repertoire of prototypes and apply them in specific design situations, and allow the designer to discover and reflect on their design knowledge

Videogame Design Support (aka Interactive Character Authoring (Design?) Support: Nelson focuses on all these design questions because he is helping design games. I’m focusing on authoring, so should I focus on authoring questions? But what about “designing” the character that you’re authoring? Are those two different things, or one in the same?
Nelson says, for games, you design (game mechanics) to produce gameplay (challenges, rewards, decisions encountered by a player). But design questions are posed in terms of gameplay first. As an interactive character author, what do you do? Code behavioral decision-making to produce reasonable actions by the actor? Write performances such that actions by the actor are readable and meaningful? Author whole scenarios to demonstrate the last two? There are SO MANY things at play that it is very hard to find a simple translation to my domain.
Nelson’s work helps to do two main things: “find sequences the designer thought shouldn’t be possible” and “query for states or sequences of events meeting some criterion”. We certainly want our tools to help find sequences that the designer/author thought shouldn’t be possible. We also want to find those that are inappropriate (violate norms) unless that was intentional.
What is the natural visual representation for an interactive character, which is fundamentally a procedure and performance spanning a potentially infinite possibility space? (to use another borrowed mangled quote from Nelson). How do we make non-graphical concepts or ideas read graphically?
We’ll probably need to borrow heavily from drama, theater, and cinematography language for individual performances, as well as game design terminology for the possibility space.

Interview Methodology: Process Mapping. I’ll talk about it more when I’ve done it a bit!

Case Studies: ABL, FATIMA, and some thing(s) from ICT. Still need to contact those folks.

Conclusions: THIS IS HARD

Future Work: Making a tool.


If you couldn’t tell, I ran out of time before my meeting. Still got a ways to go, but I’m certainly making good progress I think! I came up with three discussion questions to talk about during Pod this week with my advisers. Wish me luck!

Posted in Uncategorized | Tagged , , , , | Leave a comment

Informal Corner 10-15

This whole blog is informal, but this is more of an aside than anything.

I left my notes on campus last week when I got sick and didn’t bus up to campus again, so the stuff I wanted to talk about is both very delayed and not with me at the moment. Instead, I’m going to brainstorm and jot down ideas for my meeting with Larry tomorrow to talk about a short-term design tool idea for IMMERSE. It’s like a pincer attack, coming at the problem from two ends: an immediate, actionable piece of software we can build to make authoring better, and a highly theoretical discussion of what makes up an effective authoring tool for interactive characters.

Last week, Larry presented what I thought was my work cut-out for me! A rough design of pie-in-the-sky authoring help for the immediate future. While Andrew tried to tear the thing to shreds — repeatedly saying that it would take too much time and that it would never be used because it wasn’t doing anything useful that programmers didn’t already have a back-door for — Larry was making an effort to do what I’m going to be trying to very soon. Larry didn’t share the google doc with me, and I still left my notes on campus, but I remember it roughly. It was made of three main pieces:

  1. Checkbox control for various behaviors to allow them to trigger or not trigger: mainly for demo purposes (“Compare the behavior here … with this happy wrap-on enabled here …”)
  2. Configurable pre-launch states to not only get the author to the point at which their behavior matters/will trigger (which should be covered with Rich’s save states coming in some undetermined far-fetched future), but to specify some kind of path to test various conditions quickly
  3. Able to execute behaviors on command with various arguments

What I don’t think Larry knew much about was that 3 already existed in a very primitive form that Mike put together. The problem was that it took just about as much time to set up as it did to get to that behavior in running the program (so it was only really useful for multiple tests), and the noise/situation of the surrounding behaviors was so complex that it was very rarely ever used because the behaviors you’d trigger either a. would never happen naturally in that scenario anyway and b. wouldn’t work because of improper set-up. So I guess I can see why Andrew was poo-pooing the idea of these tools: the one tool we tried to make was time-consuming, insanely buggy, and was never really used because of all that.  Really, all we’ve determined is that 3 is going to take much more time and effort than our initial prototype to be really useful, and even then only questionably so since we’ve gotten along without it.

Ew, okay, that’s a bad response. “Gotten along without it.” Yeah, well Narratoria halved the production time of ICT’s authoring efforts. Over months, that saves a LOT of time for upfront investment. Authoring tools are very useful, but they are precision shots: they need to be like an acupuncture needle, sliding in harmlessly and seamlessly among the other production tasks to strike at a problem point and relieve pressure throughout the whole system. I need to outlaw that “gotten along without it” excuse as a dismissive and not helpful line of thinking compared to more helpful lines of critique:

  • Upfront investment
  • Disruption of other tasks
  • Solving a crucial problem.

Aside aside, the check boxes idea is simple and easy to implement, but doesn’t help authoring so much as demoing. It could be seen as a simple way to start toward part 2 too. A worthwhile investment, but a bit shallow. So we have two radical ends of the spectrum with propositions that chip away at the problem rather than strike true.

Time to talk to Larry to hear these ideas in more detail, figure out which ones are going to get implemented, and where to go from here.

Posted in Uncategorized | Leave a comment