Symbolizer
Symbolic Model-free Task Planning with VLMs
The research thesis
Ground what the world contains. Express what the task requires. Let systematic search find the route between them.
SYMBOLIZER grounds images and natural-language goals into structured symbolic representations, then uses systematic search to plan with a simulator as a black-box transition model.
Robotic and visual planning settings often provide images or text rather than a ready-made symbolic state and action description. SYMBOLIZER separates the perceptual task of grounding the current situation from the combinatorial task of finding a plan.
The accompanying paper evaluates object, predicate, and goal grounding alongside planning success across synthetic, rendered, and benchmark settings. See the paper for the complete protocol, tables, and comparisons.
The symbolic interface
Ground observations and natural-language goals into symbolic representations for planning.
SYMBOLIZER separates visual-language grounding from planning: it infers objects, predicates, and goals, then searches the induced symbolic state space through simulator transitions.
-
01
Object grounding
Infer the objects in an image or textual observation from provided examples.
-
02
Predicate and goal grounding
Express relations and task requirements as symbolic predicates over those objects.
-
03
Symbolic planning
Use the simulator as a transition model while systematic search explores successor states.
Planning with black-box transitions
Search grounded symbolic states through simulator transitions.
The simulator acts as a transition model. SYMBOLIZER can therefore formulate a planning problem from grounded symbols and explore it with classical search, without manually encoding the simulator’s actions or effects.
Grounding quality and planning performance are evaluated separately: object, predicate, and goal extraction are compared with ground truth across custom domains and external benchmarks, while planning evaluation asks whether those representations support valid plans through symbolic search and simulator transitions.
Paper / citation