[Feature Request] Support more fidelity in observations

[c-salmi]

🚀 Feature Request

Add support for observations in the form of occupancy maps, obstacle semantics, semantic maps, or even direct camera / lidar data.

Motivation

Recent local motion planning methods aim to base their motion on observations that contain more information, sometimes even direct sensor data. To include these methods we should consider adding support for more fidelity in the observations.