A GNU/Linux Wristwatch Videophone

Videophone wristwatches are a science-fiction concept that is here today. The two key inventive concepts that make this new technology possible are:

Concomitant Cover Activity

In photography (and movie and video production), it is often desirable to capture events in a natural manner with minimal intervention or disturbance. In the present invention, the scenario to be considered is that of face-to-face conversation between two individuals, where one of the individuals wishes to either make an annotated video diary of the conversation or capture a picture of the other individual, without disrupting the natural flow of the conversation.

Current state-of-the-art photographic or video apparatus creates a visual disturbance to others and attracts considerable attention on account of the gesture of bringing the camera up to the eye. Even if the size of the camera could be reduced to the point of being negligible (e.g., no bigger than the eyecup of a typical camera viewfinder, for example), the very gesture of bringing a device up to the eye is unnatural and draws attention.

The basic principle of the invention is that a camera is concealed in a device that has another purpose, where the other purpose of the device involves looking at a display. This other non-picture-taking purpose of the device provides for a concomitant cover activity. Because the camera and viewfinder (XF86 screen) are concealed in a wristwatch, when the user wishes to take a picture, he merely pretends to check the time on the wristwatch. Checking the time is a concomitant cover activity. The camera is mounted in the watch in such a way that it points out in front of the user, so a person the user is talking to will be in the camera’s field of view while the user is pretending to check the time. Moreover, the user can pretend he has trouble focusing on the watch (as if a person who normally wears bifocals has forgotten his eyeglasses) and hold the watch some distance out from his eyes, so that it will be very close to the photographic subject.

The concomitant cover activity may also involve something as simple as resting the arm on a countertop while wearing the GNUX wristwatch.

A VGA screen is configured using XF86Config set to 640×480, 24-bit colour, allowing video to be displayed at the captured resolution. The camera also operates at 640×480 resolution with 24-bit colour video capture at 30 frames per second. Images may be processed or stored locally, or transmitted at a lesser rate. A future version will transmit images at the recording speed of 30 frames per second rather than the current six to eight frames per second limit imposed by the slow (2.4 megabits per second) radio link.

The wristwatch provides a computer output screen with XF86, upon which the viewfinder function operates, using one of the windows or the root window. Graphics, including a transparent oclock, appear over the top of the video viewfinder window.

Using the VideoOrbits image stabilizer, it takes pictures at 640×480, 24-bit colour, up to 5000 pixels across, in true 48-bit colour (convertible to 24-bit colour pictures suitable for high-quality prints). VideoOrbits is available under the GPL from http://wearcam.org/orbits/.

In Figure 1, Eric Moncrief is shown wearing the watch, and Stephen Ross is pictured on the XF86 screen as a 24-bit true-colour visual.

A SECRET function, when selected, conceals the videoconferencing window by turning off the transparency of the oclock, so that the watch then looks like an ordinary watch (showing just the clock filling the entire 640×480-pixel screen). The OPEN function cancels the SECRET function and opens the videoconferencing session up again.

One technical problem that arises from running GNU/Linux (GNUX) on a wristwatch is the input. We are experimenting with an input pie menu system. A user can easily select eight directions of the compass, but since this device has a clock face (at least, that is its concomitant cover usage), a 12-position pie menu makes the most sense.

The pie menu is described in “A Comparative Analysis of Pie Menu Performance” by Callahan, Hopkins, Weiser and Shneiderman, 1988.

Figure 3 depicts a natural choice of pie menu for a wristwatch display. The display is typically a computer screen with 480 pixels down and 640 pixels across, measuring approximately 0.7 inches on the diagonal. Upon the display is the image of a clock face, superimposed on top of a video signal from the camera. Time is displayed as a transparent xclock or oclock (or both, one superimposed upon the other). Our modified oclock is available from http://wearcam.org/gclock/ and an exclusive or (EOR) oclock is under development to reduce screen real estate use. In the figure depicted here, the time is 4:03.

The device truly looks like an ordinary wristwatch, although one in which the hands are displayed electronically, because it is in fact a wristwatch, among other things. It is natural for such a wristwatch to have a circle displayed on the screen (this is a feature of the original oclock), but unlike the oclock, it has numbers displayed around the periphery of the circle. In this way, it is easier to tell time, and the numbers may also be assigned a secondary meaning (e.g., select “0” to stop recording, “4” to kill all processes and halt the processor, “7” to wake up the system from sleep mode, etc.).

Since humans are quite good at telling time, the numbers are often missing from commercial wristwatches, and some wristwatches do not even have markings for each hour. Instead, we often rely on our heightened sense of visual acuity to discern the angle of the hands upon the clock face. Thus, it is no surprise that the clock menu is usable without paying much attention to the face of the clock. The user just needs to stroke the face of the clock in the direction desired.

The entry of numbers on a touch-sensitive clock face in the context of the current invention may be done as vectors (e.g., with no regard to location, only to direction). Thus, a stroke from left to right is regarded as the number 3, regardless of where the stroke begins or ends. A downward stroke (e.g., from top to bottom) is regarded as the number 6 regardless of where the stroke begins or ends, and so on.

Thus, telephone numbers can easily be entered into the device, and similarly, an alphabet can be constructed much like the alphabet of an automated DTMF (dual-tone multi-frequency) answering system used for voice mail and the like in telephony.

Since there are 12 pushbuttons on a telephone and also 12 hours on a clock face, there can be a one-to-one correspondence between the numbers of the clock face and those of the telephone. The hours 10:00 and 11:00 are used for the symbols “*” and “#” of the telephone touchpad.

The data entered by way of the clock face menu is typically combined with the video recording made from the scene. The clock face menu is sufficient for entering a department store manager’s name, which may be appended to the video file header, so that a large database of recorded video may be navigated later using these short text headers.

Due to direct contact between the wristwatch and the body, the pulse (heart rate) as well as skin conductivity (sweatiness index) of the wearer may be determined, and this information may be appended to or recorded with the video signals. This may facilitate, for example, a future search through all video in which the wearer’s heart rate exceeds a certain threshold. It has been found that when a department store manager is dishonest with respect to refund policies, or a clerk refuses to tell a customer his name, the customer’s heart rate increases dramatically, and the customer often sweats profusely. Thus, this extra information can later help locate moments of tension in a previously recorded argument at the refund counter.

Linaccess: GNUX For Low Vision

G Linaccess (Accessible GNU/Linux) is GNUX for the partially sighted or visually challenged. The purpose of the G Linaccess project is to make computers accessible to those with low vision, in a way that does not require the use of a text-to-speech converter or other devices normally used by the blind.

With G Linaccess, those with low vision can still enjoy the visual aspects of computing through the use of enlarged fonts, which are optimally designed in contrast and shape to be readable, despite visual defects that may prevent the person from reading traditional large-print books and the like. Moreover, through the use of OCR, people can acquire the text of articles and consume this material on the screen of a cathode-ray tube or laser EyeTap device by virtue of its much higher contrast ratio than that of paper and other print media used for large-print books.

The first phase of the G Linaccess effort has been the design of an XF86Config file suitable to the visually challenged. Next, I put together a simple set of commands including xterm40, xterm30 and xterm20 to allow for creation of large-font xterms. The window manager is also configured for easy movement from one window to the next without using a mouse. An important consideration was the decision to not use a mouse because of the visual dexterity, visual attention, etc. that pointing devices tend to demand, and also the difficulty of implementing a good pointing device on a wristwatch.

Various window-manager files are included, the preferred being .fvwm2rc for the FVWM2 window manager. Pointerless (mouseless) operation is facilitated in each of these.

An .xinitrc file is also included, which executes FVWM2 in the background. A clock, rather than the FVWM2, is the last item. This makes it easier to kill the window manager and start a different one. It is assumed that if a wristwatch were to have only one window remaining after you killed off all the others, the last to go should be the clock.

The clock is large but transparent, so it is easy to read without blocking too much of the screen (because of its transparency). An exclusive or (EOR) clock is in the works, and will provide for zero occlusion so that users can enjoy a large easy-to-read clock that doesn’t consume any workable screen real estate.

The choice of colours has been a white foreground and a black background for best readability.

I have four students now who are helping with this project and re-writing many of the GNUX commands to operate on a reduced-character screen. We’re getting these to work nicely on a 30×12 text screen. An accessible FreeSource BIOS is also planned for later.

To use G Linaccess:

• Download the file http://wearcam.org/glinaccess.tar.gz
• Unpack it, e.g., tar -xzvf glinaccess.tar.gz (or if your operating system doesn’t support uncompression in tar, then gunzip glinaccess.tar.gz and tar -xvf glinaccess.tar).
• This should create a directory from your home directory called ~/glinaccess which should contain various files.
• Copy the given .fvwm2rc and .xinitrc files into your home directory. Back up the old ones you already have there, if you want to keep them.
• Start the X Window System.
• Now you should see x20, x30, x40 and x80 windows on your screen, as well as a very easy-to-read clock.
• You should be able to move between these windows by holding down the ALT key and pressing N and P to go to the next or previous window, until you get to the clock, which defines the end of the window list.
• Start a bash (Bourne Again) shell, or the shell of your choice.
• Source glinaccess/glinaccess.sourceme.
• Now you should be able to use any of the commands in ~/glinaccess.
• For example, the x30 command will create a 30×12 window, with what I have found to be the best choice of font. Of course, your supported fonts may vary from system to system, so you may want to experiment with the sizes of the fonts.

Here is a list of the window-creation commands I support and the X window sizes each produces:

• x10 is 10 letters across and 4 down.
• x20 is 20 letters across and 8 down.
• x30 is 30 letters across and 12 down.
• x40 is 40 letters across and 15 down.
• x47 was created because the font looks good on my rig.
• x56 was created because the font looks good on my rig.
• x62 was created because the font looks good on my rig.
• x80 is 80 letters across and 24 down.

x47, x56 and x62 are arbitrary sizes that I created by experimenting with different fonts. On my computer, I found they were very readable on a variety of tubes and eyetap devices. x80 is just a normal window, and is included so that a non-visually-challenged user can use this system, since so much of the world is based on the traditional 80×24 window size.

A number of non-visually-challenged users are using linaccess, simply because it results in less eyestrain and is more comfortable for long-term use. In this case, a mixture of different window sizes are used (x80 where needed, and others such as x40 where there’s no particular formatting of text). Presently, most of linaccess is prepared for an x30 window (e.g., 30 letters across and 12 down). If you type the x30 command, you should get a nicely readable window. If the x30 window has strange spacing or doesn’t work with all the other programs, try changing the font size by just one number. In particular, change -*-courier-bold-r-normal--32-0-*-*-*-*-*-* to -*-courier-bold-r-normal--31-0-*-*-*-*-*-* or -*-courier-bold-r-normal--30-0-*-*-*-*-*-*. On some systems, the available fonts vary, and certain ones don’t produce well-formed letters. You will need to experiment until the letters are well-formed and produce as little eyestrain as possible.

The small size of the display required the development of an X Window System configuration that was easy to read on a small screen. This gave rise to the Linaccess project, where GNUX was made accessible on low-vision systems. This project has two distinct but closely related goals. First, to make GNUX accessible to the visually challenged, or those suffering from low vision, such as age-related macular degeneration, glaucoma or the like. (Note that this project differs from the blinux project in the sense that the goal of Linaccess is to use visual output, but to make it more accessible to those with low vision.) Second, to make GNUX usable on small screens, such as the wristwatch system. In many ways, we’re all suffering from low vision when we’re trying to read a 0.7-inch diagonal screen.