[0001] 1. Field of the Invention
[0002] The present invention relates to observing and recording the interaction between a person and a sophisticated video device (e.g. a digital set top box) using an infrared remote control.
[0003] 2. Background Art
[0004] In providing an efficient and satisfactory video service such as digital cable or satellite television, it is essential that providers are able to understand and cater to the needs of users. In order to accomplish this, observation systems for monitoring and recording interactions of persons with sophisticated video service devices such as digital set top boxes are used.
[0005] Existing processes for performing this type of observation involve having the person complete a paper log showing what they watched and when or involve the use of multiple devices such as video cameras, video tape recorders, and video mixing consoles in order to acquire information. The paper log approach is limited by a person's motivation and memory. However, in the multiple device approach time synchronization among the different data acquisition devices can be a problem. Also, not all types of user events are recorded and, as a result, the variety of analysis may be limited. Further, video service devices implement a complex collection of interactive menus. The full range of possible transitions from one menu to others will not generally be known in advance.
[0006] For the foregoing reasons, there is a need for an improved method of monitoring and recording user interaction with a video service device.
[0007] It is therefore an object of the present invention to provide a computer-based method of monitoring and recording user interaction with a video service device wherein events from multiple devices are recorded to the storage medium as a sequence while preserving the time relationship among the various events with respect to each other to thereby establish a common time base for the recorded sequence of events.
[0008] In carrying out the present invention, a computer-based method of monitoring and recording user interaction with a video service device is provided. Relevant information is recorded to a storage device as a sequence of events that maintains the time relationship among each recorded piece of data. The storage device may be part of the computer or remote storage. The computer is further equipped with a first video frame grabber, and a second video frame grabber. The first video frame grabber is connected to the video service device while the second video frame grabber is connected to a video camera. A remote control signal detector is connected to the computer. The video service device may be any device (for example, set top box, personal video recorder, video tape recorder, etc.) so long as direct or baseband video output is provided to the first video frame grabber. In this way, the first video frame grabber is directly monitoring the same video that the person is watching. Further, the invention comprehends that, in the same way, multiple video service devices may be monitored so long as the appropriate direct or baseband video output is provided to the first video frame grabber.
[0009] Remote control signals indicative of remote control button clicks by the user are detected over a period of time. A plurality of first video frames are grabbed by the first video frame grabber from the video service device. A plurality of second video frames are grabbed by the second video frame grabber from the video camera. The information including the detected remote control signals, the first video frames, and the second video frames is then recorded to the storage device as a sequence of events over a period of time while maintaining the time relationship among the various events with respect to each other and thereby establishing a common time base for the recorded sequence of events. The remote control signals may be presented and recorded as raw signals or may be interpreted and the resultant interpretation recorded.
[0010] At a more detailed level, the invention comprehends additional features as further explained below. These additional features are optional.
[0011] In order to increase the probability of capturing adequate and useful information for analysis, a sequence of first video frames is grabbed by the first video frame grabber from the video service device in response to a remote control signal indicative of a remote control button click being detected by the remote control signal detector. If a remote control signal is detected by the remote control signal detector while the first video frame grabber is grabbing the sequence of first video frames, a counter is restarted such that the grabbed sequence of video frames extends beyond the additional remote control signal detection. The sequence of first video frame grabs is then recorded to the storage medium. That is, in the case of a single click, one frame is captured immediately and additional frames are captured at predetermined intervals to form the sequence. The additional frame capture schedule starts over if another button click occurs before the scheduled captures are completed.
[0012] Capturing images from the video service device is helpful when tracking user interaction with the service control software. It is not sufficient to record just remote control events because depending on where the remote control signal detector is positioned relative to the service delivery device, a signal may be detected by one device but not the other. Some digital set top boxes are known to ignore subsequent remote control signals during intervals while processing an event. Recording the menu progression as a sequence of images makes subsequent analysis of user actions feasible. Captured images from the video service device may be used to determine the tuned channel by capturing a sequence of channel grid screens or capturing an image of the flip bar that appears when a channel change occurs.
[0013] It is also comprehended by the invention to capture snapshots of the user interacting with the video service device. A new video frame from the camera is grabbed by the second video frame grabber at regular intervals. In certain circumstances, the latest video frame is recorded. In particular, the latest video frame is recorded if a remote control event occurred between the previous snapshot and the current snapshot or if the current snapshot is significantly different than the previous snapshot.
[0014] In determining a motion differential between consecutive snapshots or frames, the particular second video frame grabbed by the second video frame grabber from the video camera is compared with the immediately preceding second video frame. The motion differential criterion acts as a form of motion detection to allow the recorded information to aid in the analysis of patterns of motion in front of the television relative to the pattern of remote control events.
[0015] The observation system may be implemented in software running on a computer such as an ordinary PC hardware and the Linux operating system. As such, in addition to the above described methods, the invention further comprehends a computer readable storage medium having instructions stored thereon to direct a computer to perform monitoring and recording of user interaction with the video service device. Of course, the computer hardware and operating system may be any hardware and operating system that are appropriate.
[0016] The above object and other objects, features, and advantages of the present invention are readily apparent from the following detailed description of the preferred embodiment when taken in connection with the accompanying drawings.
[0017]
[0018]
[0019]
[0020] Referring to
[0021] The system
[0022] Remote control signals are detected by the remote control signal detector
[0023] With reference to
[0024] Video frames from the video service device are stored in response to button clicks on the remote control. For each remote control signal detected by the remote control signal detector
[0025] Video frames from the camera are stored in response to a button click preceding a particular video frame and also in response to significant motion between successive frames. For example,
[0026] In addition, numerous other features could be implemented to enhance system functionality. Continuous frame grabbing and a pattern recognition process could be used within the observation system
[0027] Additionally, video frame grabs from the video camera
[0028] Since the video frame grabs by both the first and the second video frame grabbers are under software control, it is possible to implement a wide variety of rules and conditions for recording the video frame grabs. For example, time of day sensitivity could be used, and motion sensitivity could be increased or decreased after remote control button clicks. The lapses between regular video frame grabs from the video camera
[0029] Although
[0030] Since the exemplary observation system runs on ordinary PC computer hardware and Linux, it is easy to add network access to field-deployed boxes. Such access could be used to enable a near-real-time analysis and essentially unlimited recording capacity.
[0031] This invention makes it possible to unobtrusively observe users in a home-like environment while interacting with sophisticated video service devices. Such observations are necessary in order to determine how users employ the various capabilities of the video service device. Previous observation systems consisted of multiple parts and were not capable of capturing the entire range necessary for detailed analysis. Through the application of this system, experts could significantly improve the user interface design of software for sophisticated video service devices such as digital set top boxes and personal recorders. Further, increased understanding of how and when subscribers use such devices and the program menus could result in tailoring of video services to make them more attractive and competitive.
[0032] While embodiments of the invention have been illustrated and described, it is not intended that these embodiments illustrate and describe all possible forms of the invention. Rather, the words used in the specification are words of description rather than limitation, and it is understood that various changes may be made without departing from the spirit and scope of the invention.