*3.4. The Proposed Framework*

The proposed framework for task and motion planning under uncertainty extends the basic *Contingent-FF* planner, aiming to incorporate different geometric reasoning procedures, observations on sensing actions, as well as human–robot collaboration within planning. The overview of the system is sketched in Figure 1. It involves three main parts: *Heuristic Computation, Space Search*, and *Conditional Plans Evaluation*.

*Heuristic Computation* basically provides a value which is distance to goal and promising actions for each belief state. The basic *CRPG* is initially computed and the associated relaxed plan is obtained. This plan is forwarded to the relaxed geometric reasoner determining the feasibility of actions in terms of reachability, collisions, manipulation constraints, and graspability. The heuristic value is returned along with helpful actions if such constraints are met. If a constraint is violated, the associated belief state is updated with facts describing the cause of failure, and an alternative relaxed plan is looked for. Hence, the heuristic function is informative both in terms of symbolic and geometric constraints.

*Space Search* maintains the basic algorithm of the *Contingent-FF* planner that is based on the *And-Or* search strategy. From each belief state, the action resulting in the state with lowest heuristic value is selected and is a candidate to be added to the conditional plans. The only difference is that the heuristic value now accounts for geometric constraints.

*Conditional Plans Evaluation* tries to solve motion planning for an action if possible. It considers sensing procedure to evaluate sensing actions and may assign actions, which are infeasible for robots, to operators.

If motion planning fails, the current belief state is updated with the cause of failure and the search resumes. In general, the geometric failure could be due to collisionable objects, so the literal *isCrit* is added to the belief state. If the failure is because of fixed obstacles blocking the way of reaching an object, inverse kinematic problems, or motion planning time-out problem, the literal *infeasByRob* is added to the state. In such failure, if the type of the evaluated action is either transit or open, the literal *assist* is also inserted.

Otherwise, when motion planning succeeds, the action is added to the tree-shaped conditional plans at hand. After finding the complete conditional plans, feasible actions are executed by a robot or a human operator in the real world and sensing actions are observed either using a perception module or the information provided by the human operator in run-time. Therefore, the robot plan can determine the correct branch to follow up its plan.

**Figure 1.** The proposed system overview of contingent task and motion planning using the extended version of *Contingent-FF*.
