I don’t think that’s quite true. Of course it would make things easier, but there are loads of applications where a smartphone “controller” would work.

Like setting a route on Google maps and having navigation steps shown on your glasses. Doing hands free video calls. Live translation of a conversation. Or simply notification popups.

As long as it’s an ongoing process, you simply take out your phone, start the app/program, and voilà! It would just be more of a display until we develop a neural interface.