作者认为,AI的人机交互界面正从当前的Markdown默认输出向更丰富的HTML格式演进,后者能提供更好的图形、布局和交互性。长远看,由扩散神经网络直接生成的交互式神经视频或模拟将是终极输出形式,但其如何与精确的传统软件结合仍是开放问题。在输入侧,仅靠音频、文本或视频并不足够,需整合如手势指点等更自然的交互方式。总体而言,在迈向脑机接口之前,人机输入输出的融合仍有大量工作要做。现阶段的一个实用建议是尝试让大语言模型以HTML格式结构化其回复。
This works really well btw, at the end of your query ask your LLM to "structure your response as HTML", then view the generated file in your browser. I've also had some success asking the LLM to present its output as slideshows, etc.
More generally, imo audio is the human-preferred input to AIs but vision (images/animations/video) is the preferred output from them. Around a ~third of our brains are a massively parallel processor dedicated to vision, it is the 10-lane superhighway of information into brain. As AI improves, I think we'll see a progression that takes advantage:
1) raw text (hard/effortful to read) 2) markdown (bold, italic, headings, tables, a bit easier on the eyes) <-- current default 3) HTML (still procedural with underlying code, but a lot more flexibility on the graphics, layout, even interactivity) <-- early but forming new good default …4,5,6,… n) interactive neural videos/simulations
Imo the extrapolation (though the technology doesn't exist just yet) ends in some kind of interactive videos generated directly by a diffusion neural net. Many open questions as to how exact/procedural "Software 1.0" artifacts (e.g. interactive simulations) may be woven together with neural artifacts (diffusion grids), but generally something in the direction of the recently viral https://x.com/zan2434/status/2046982383430496444
There are also improvements necessary and pending at the input. Audio nor text nor video alone are not enough, e.g. I feel a need to point/gesture to things on the screen, similar to all the things you would do with a person physically next to you and your computer screen.