The episodic memory consists of a large database of previously analyzed and executed Scene Graphs, Scenes, Scripts and Plans, together with additional information including the "context" (other scenes) and the information if plan execution were successful or not.
This database is used by most TinyCog Subsystems to retrieve information for specific purposes. In this sense, the episodic memory is a merely passive database with specific query mechanisms designed to serve the requirements of the active Subsystems:
- 3D Reconstruction
- Retrieves 2D images from the episodic memory based on image features including edges, textures etc. in order to take advantage of past confirmed object classifications.
- Object Configurator
- Retrieves 2D images of individual objects from the episodic memory with specific internal states.
- Plan Recognition
- Retrieves Plans from the episodic memory based on individual actions, objects or agents occurring in these plans.
- Prediction Subsystem
- Plan Reasoning
- When optimizing plans and choosing different plan variants, the plan reasoning subsystem will look at past performance of plans. This can be implemented by machine learning algorithms clustering properties of the initial Scenes of a plan depending on the success.
- Attention Subsystem
We assume that the episodic memory will have to store and index massive amount of data, so that storage and indexing will be distributed across multiple server systems.
Comparison and References
TinyCog 0.0.1 does not yet implement an episodic memory. However, this will become necessary as when implementing the probabilistic Planner.