When one does software development, one might use a variety of media. Technical pen on paper for design-space and architecture graphs. Whiteboard. Whiteboard with colleague. Notes. Exploratory code. And so on. So one question to ask is, here comes consumer VR and AR, and what might these new media offer to justify their place in this tool collection? That takes exploration.
Another question to ask is, given hardware with 3D tracking and display, how might you usefully tweak your existing tools to leverage that?
Rather than reinventing wheels, languages, keyboards, etc, I like the approach of augmenting what exists. And I use emacs a lot.
TensorFlow posenet-based head tracking is easy.[1] And wiggle 3D, and anaglyph shaders, aren't hard. And making DIY shutter glasses (arduino, phototransistor, and Adafruit light valves) was surprisingly simple. So my generic thinkpad screen can do jittery, washed-out, or flickery 3D. And electron/node.js/v8 can serve up low-latency video of the desktop, to be sliced and diced. So an app can provide two or three eye views, in separate windows or time multiplexed, and it can be shown as 3D. When last I tried to do that separation in emacs, I timed out, still unsure if there was a viable path. Thus my query.
Another thing one might try is to overlay graphics. An emacs could have an arbitrary backchannel chat with the compositor, and draw 3D lines and such on itself. And of course control its own window positions and orientations, fragmentation and coalesce. ;)
If you have red-cyan anaglyph 3D glasses, you might enjoy toying with https://atom.io/themes/anaglyph-syntax - but trading color space for depth is a steep price.
> What would you use this for?
So basically for exploration. What might our tooling now look like, if we'd had 3D display and tracking in widespread use for the last two decades?
Another question to ask is, given hardware with 3D tracking and display, how might you usefully tweak your existing tools to leverage that?
Rather than reinventing wheels, languages, keyboards, etc, I like the approach of augmenting what exists. And I use emacs a lot.
TensorFlow posenet-based head tracking is easy.[1] And wiggle 3D, and anaglyph shaders, aren't hard. And making DIY shutter glasses (arduino, phototransistor, and Adafruit light valves) was surprisingly simple. So my generic thinkpad screen can do jittery, washed-out, or flickery 3D. And electron/node.js/v8 can serve up low-latency video of the desktop, to be sliced and diced. So an app can provide two or three eye views, in separate windows or time multiplexed, and it can be shown as 3D. When last I tried to do that separation in emacs, I timed out, still unsure if there was a viable path. Thus my query.
Another thing one might try is to overlay graphics. An emacs could have an arbitrary backchannel chat with the compositor, and draw 3D lines and such on itself. And of course control its own window positions and orientations, fragmentation and coalesce. ;)
If you have red-cyan anaglyph 3D glasses, you might enjoy toying with https://atom.io/themes/anaglyph-syntax - but trading color space for depth is a steep price.
> What would you use this for?
So basically for exploration. What might our tooling now look like, if we'd had 3D display and tracking in widespread use for the last two decades?
[1] https://storage.googleapis.com/tfjs-models/demos/posenet/cam...