United States Patent 3846826

This invention enables a person to paint or draw directly into color television. No special probe or stylus is required since a person can use brushes or pens, fingertips, rubber stamps, or any drawing or painting object whatsoever. At the same time, a person can play his free hand over a piano-like keyboard to synthesize images by manipulating or altering the images or forms as they are introduced. It is applicable to graphic productions of all sorts, computer input-output graphic processing systems, for visualizing mathematical transformations, or for use with scanning lasers or electron microscopes that etch or score.

Application Number:
Publication Date:
Filing Date:
Primary Class:
Other Classes:
345/22, 345/168, 345/180, 348/239
International Classes:
G06F3/033; (IPC1-7): H04N9/02
Field of Search:
View Patent Images:
US Patent References:
3340401Motionless data input key1967-09-05Young
2168049Electro-optical system1939-08-01Skellett

Primary Examiner:
Murray, Richard
Parent Case Data:

This application is a continuation-in-part of U.S. application Ser. No. 171,155 filed Aug. 12, 1971, now abandoned.
What I claim is

1. In a real-time television drawing system comprising

2. A system according to claim 1,

3. A system according to claim 2 wherein the keys of said keyboard vary the magnitude of the voltages proportionately to the pressure applied to the keys.

4. A system according to claim 3 wherein said keyboard actuates a sound producing instrument.

5. A system according to claim 1 wherein said display system includes a color television display device, and means for supply color television signals to said display device in response to said electrical signals.

6. A system according to claim 1, including:

7. An image producing system, comprising

8. A system according to claim 7, wherein said means for producing an energy beam includes a microscope wherein said scanning beam physically changes points within the microscopic area.

9. A system according to claim 7, wherein said last mentioned means includes apparatus for producing a color television image.

10. A system according to claim 7;

11. In a real-time television drawing system comprising;


Ever since television was invented artists and designers have been trying to discover exactly what qualities are unique to it as an artistic medium. Recent experimentalists have begun to create images directly in the video medium by electronic circuit manipulation. My invention is a first step toward channelling video image manipulation, allowing a person the necessary freedom of control to keep the human imagination in a free interaction with the medium as unrestrained as possible. It eliminates the need for special styluses or probes, and allows an artist or designer to use the most subtle of his tools directly: the paint brush or a fine steel penpoint. Thus a person can draw or paint directly into the medium of color television in full color without the use of pigments, dyes or inks.

The invention operates when a person places any type of a drawing or painting instrument on a special paintscreen. Marks appear on the color television monitor at points or areas corresponding to the shape of the drawing object tip. One by one, as they are introduced by hand, these marks are made any desired color by controls. The marks can remain where they are placed for a short or a long time, or they can be made to disappear electronically. Color mixing between successively placed forms is either additive or subtractive, or the new color can dominate the old, accomplished electronically and under control of the person's free hand.

Further, the invention enables a person to exert a wide range of automatic visual controls to synthesize images from those actually drawn. For example, by switching on a circuit when a key is pressed it is possible to cause instant size changes in all images drawn in. Other systems allow a person to create the letters of the alphabet, so that he can set them down into the television screen, in any desired color, and move them about at will until he has composed his image.

The versatility of this new medium will obviously make it an important artistic medium for commercial or fine art applications. The results of using this invention can be experienced by looking at the color television monitor at the same moment images are made, or they can be broadcast live over a television station. The pictures can also be recorded on video tape or moving picture film for later viewing. The painting can also be a collaborative effort in which two or more of the instruments are connected together electronically. My invention is in fact a device that turns the visual arts into performing arts so that duets or trios, or even orchestras of visual productions, are possible.

It is clear that my invention has four main areas of application. First, as mentioned above, it is an extremely useful tool for art studios or creative artists, for making pictures for advertising or other printed media, or making animated motion pictures. When used for film animation, background scenes or previously drawn pictures can be applied from conventional television inputs, appearing under the drawn images on the output monitor.

Secondly, the invention serves as a new computer input device, having certain abilities to pre-process forms so that less analysis will be required by the computer, simplifying the problems of pattern recognition and assimilation by the computer.

Third, the invention leads to a simple way for mathematicians to visualize and experiment with electronic symbols, or symbols that immediately manifest mathematical properties such as visual transformations, as for example in transformations that occur when passing from the real to the complex number plane.

Fourth, the output of the invention can be used to control a scanning laser or microscope system employing scanning electrons, for the purposes of etching or burning, allowing a person to perform delicate manual burning or scoring over areas, even down to microscopic dimensions.

Other objects will become obvious as the details of my invention are described. For example, the images can be created by means of musical notes, filtered from an audio circuit and made to activate the manipulating circuits automatically. Various types of image synthesis can also be achieved by differing circuit arrangements, using various types of feedback, for example. Depth can be simulated by causing successively memorized images to be of differing sizes. Depth can also be created by doubling the circuits, providing a means to draw in a volume, and providing dual monitors with appropriate optical systems that allow the right and left eyes to differentiate the images created.

The resolution or detail of all images created in the invention can be made as sharp as desired since it is a closed-circuit color television system. If it is desired to convert the output into standard interlaced 525-line images, they can be derived from a scan converter connected to the output system, or by imaging the visual output directly with a studio television camera.

These are only some of the ideas and applications of my invention that come to mind, but the spirit of my idea as related in detail here naturally include many more.


Other objectives will be apparent from the following detailed description of a preferred embodiment of my invention, taken in conjunction with the accompanying drawings. In the drawings:

FIG. 1 shows the optical system whereby hand-placed marks on a prism cause an interruption of a flying-spot scanner raster image and create a "paintpulse" gating signal.

FIG. 2 shows a block diagram of a system for utilizing the paintpulse gating signal to introduce and retain any arbitrary color on the screen of a closed-circuit color television system.

FIG. 3 indicates the system whereby movements or distortions can be introduced into the raster to affect the final image.

FIG. 4 shows the details of a key on a piano-type keyboard used to activate and control manipulative image changes through a potentiometer and microswitch arrangement.


Painting or Drawing in Color Television

There are many ways to convert a manual movement into an electronic signal suitable for use with a television system to uniquely establish points over the picture area. Most conventional systems use a special probe that a person places onto a television screen, either a probe that detects a position signal from the television raster light itself, as in U.S. Pat. No. 3,271,515 by T. P. Harper, or a probe that causes a position signal to be generated from an overlay or a deformable surface placed over the television screen, as in U.S. Pat. No. 2,227,083 by E. Hendrick. In general a probe has two major disadvantages: it has an inflexible point without variety, and it defines only single points at a time. This limits its usefulness as a source of manual graphic inputs for visual ideas of the complexity required for art productions.

My system, however, enables an artist to use conventional brushes or pens for painting or drawing into color television signals. He can also use his fingers, any type of rubber or plastic form or design, like a rubber stamp, or any stylus whatsoever, which he touches to a special draw or paintscreen, that is one surface of a polished glass or plastic prism or the like. A single stylus can produce a wide variety of line qualities, as for example when its tip is flat so that it can be turned by hand as it is moved, like a flat pen point turned to draw both thick and thin lines. Or if a brush is used, it can cause a very tiny dot or line when only the tip is engaged, or a very broad line when it is pressed down. In addition, every point touched on the draw or paintscreen is introduced, so that multiple tipped styluses can introduce large shapes, or rubber stamp-like shapes can introduce symbols or letters, instantly positioned by hand anywhere on the paintscreen and therefore on the output area.

FIG. 1 shows the optical system for causing hand-introduced marks to generate a paintpulse gating signal. The raster 1 of a flying-spot cathode ray tube (CRT) 2, whose sweep is derived from a standard set of vertical and horizontal deflection generators of any line definition desired, is focussed by means of lens 3 through wedge 4 onto the internal totally-reflective undersurface of prism 5. The principal focus of the lens is indicated in the figure by the dashed lines ab to cd, of which only the side portion ae is used. The wedge 4 compensates for the prism refraction in order to bring the entire raster 1 into sharp focus and undistorted at the internal totally reflective surface of prism 5. Normally the entire light from raster 1 is reflected through the prism 5 to condensing lense 6, which gathers all the light and passes it to photo multiplier 7. When some object, like a hand-held brush or stylus 8, touches the top surface outside the totally reflective surface, called here the "paintscreen," it causes light at that point to be trapped and not reflected out. For this moment, corresponding to point A of line L in raster 1, there is no, or less signal detected by photo multiplier 7. The signal 9 will appear to dip down from white to black, corresponding to this point, as shown for that line, or for as many points in the lines that the brush or stylus 8 interrupts the raster 1 light. The pulse generated by an arbitrary point A is called the "paintpulse." The prism 5 obviously must be highly polished. A slight amount of optically pure oil on the brush or stylus will aid this effect, but there is sufficient natural oils on brushes that this is unnecessary unless a person is trying to achieve unusual painting effects, in which case he might want to charge a brush highly with water or oil and let it flow over the prism pointscreen as in water-color painting.

FIG. 2 is a block diagram showing how the paintpulse causes any desired color to be generated on the output screen of a color television system. The paintpulse is first amplified in video amp 10, and clipped in the clipper. Clipping is not necessary of course if the paintpulse is of the correct amplitude to drive the gates, as determined by the video amp. 10. Clipper 11 is a voltage-limited diode circuit. The paintpulse serves then as a gating signal to activate three gates 12, 13 and 14. They are each pentode-type keying gates, biased so that when a paintpulse signal is sent to their screen grids they will pass all signals applied to their control grids respectively. The control grids each receive signals from the color generator 15. Color generator 15 produces voltages corresponding to red, green and blue color television signals as they would be required for application directly to the control grids of a color television tri-beam CRT. Since the system is closed circuit, these signals can be dc. The signals necessary to cause white to appear on a color CRT. for example, must have the correct proportionality so that the color phosphors generate the necessary light components that add up to white, namely, equal voltages on each grid. When the three outputs from the color generator 15 maintain this relationship they will produce from black through the grays to white, depending upon their voltage amplitudes. Now if one or the other color signal voltage predominates, that color will predominate in the CRT output; and various combinations of the signals will produce all the colors in the rainbow, at all intensities and saturations, as is well known in the art. The color generator 15 generates these various voltages from a potentiometer-controlled resistive matrixing circuit which generates the R, G and B (Red, Green and Blue) signal voltages under controls 17. This matrix is arranged so that a single control element can be used to introduce the necessary voltage combinations to result in any desired color hue in the output tube. This embodiment of my invention does not require the generation and control of the NTSC composite color television signal since it is closed circuit. If it is desired to use this signal, color gen 15 is a colorizer or color encoder, and the remaining output circuits are designed to operate from this signal, as in a standard color television receiver. The color controls of color gen 15 can be hand set; they can be attached to the piano-like keyboard described below; or they can be mounted on the stylus or brush 8 so that a person can control the color as he paints. The three gates for red 12, green 13 and blue 14 are triggered only when a paintpulse is sent from clipper 11. When they are triggered, they pass the various components of the color signals from color gen 15, set by the controls 17, to three double-ended storage tubes 18, 19 and 20. The signals at this pont are now video signals due to the paintpulse gating actions through the gates, and they are stored by means of the write portion of the storage tubes, deflected across their internal storage surface by the horizontal and vertical deflection portions of scan gen 21, synchronized to the raster 1 of the original flying-spot scanner CRT 2. The various levels of the color signals establish various levels of charge across the storage surface of the storage tubes 18, 19 and 20. This storage is necessary to preserve various signals as the person paints in images across the paintscreen. The read sections of the storage tubes, also deflected by scan gen 21, detects the various levels of charge across the storage surface, and passes them from a collector element to the color control grids of the color CRT 16. The sweep circuits of the color CRT 16 are also controlled by the scan gen 21. synced to raster 1 through sync 22 in FIG. 3. Blanking signals for the color CRT 16 are generated in 43, also in sync with raster 1.

Therefore, in sum, when a person sets a brush or stylus 8 on the paintscreen of prism 5, he interrupts the raster light 1, causing a lack of light at this point A, which is detected by the photo multiplier 7, amplified in 10, and clipped in 11, and passed to gate in the color signals set by controls 17 in color gen 15, and passed to control the color grids of color CRT 16 through the storage tubes 18, 19 and 20, becomming visible and cumulatively disposed in the correct chosen color at the correct chosen spot corresponding to the original mark position on the visible output screen.

Other images can be introduced into the system through a conventional video resistance network matrix which accepts and combines various video inputs, without their influencing one another. The other images may be background images or other information to be combined with the newly painted inputs, or they may be the input of another similar system controlled by another or by more people. Finally, the images appearing on color CRT 16 can be photographed, or viewed by the person making the color video image. The tri-color output can be replaced by three black-and-white CRTs each therefore giving the red, green and blue images, that lend themselves to be photographed for creating the separation images required for the standard three-color printing process. The entire color output can also be replaced with a scanning laser, scanning electron, iron, X-ray or other scanning system, as described below.

Manipulative Potential

This system of point-by-point painting into color television lends itself to the creation of many automatic alterations on the points as they are introduced on the paint screen. My system inherently provides two general manipulative potentials, first one caused by altering, moving or distorting the original raster ground; and second one caused when the gating system introduces information in addition to the color signals, as for instance images derived and generated by vidicon cameras trained on real images or designs.

Raster Ground Manipulation

Considering the first raster ground manipulation, it is clear that the original raster 1 must be in exact focus on the prism 5 totally reflective undersurface, which was the purpose of the correcting wedge 4; that it must not be distorted; and that its shading must be uniform; or otherwise the output signal will not be correct with respect to the drawing or painting laid down on the paintsurface. This fact can be exploited to cause variations. For example, size variations in the raster 1 will be reflected as size changes in the output. If the raster 1 is made very small with respect to the paint surface of the prism, all movements of the brush or stylus will be written large in the output since small hand movements will encompass the small raster area very easily. And vice versa, if the raster 1 is made to overscan the paint surface, and only a small portion of it covers the paintscreen, all movements of the brush or stylus will only intersect a small portion of the raster, and all movements will appear as very small movements in the output since only that portion of the raster scan under the paint surface will be affected by the stylus or brush movements.

FIG. 3 shows how the various control elements of the vertical 23 and horizontal 24 deflection generators that generate raster 1 on CRT 2, can be brought out and controlled by a piane-type keyboard 25. The purpose of this piano of typewriter-like control 25 is to create a versatile and subtle means whereby a person can, with skill and training, be able to synthesize a broad variety of abstract or realistic designs or images everywhere that the painting stylus or brush is placed. The effects of three controls in the vertical circuits and three controls in the horizontal circuits, or six keys on 25, are illustrated in this embodiment of my invention. It is to be understood that the raster can also be altered, distorted, moved or otherwise controlled by means of anaspherical, anamorphic or other types of lenses or mirror systems. The raster can also be computer controlled to create many very complex variations in its geometry and image synthesizing abilities. It can also be controlled by electronic circuits to create symbols, simply by causing it to be moved about in toto by signals that form these symbols and are applied to the centering voltages of the CRT 2, under keyboard or other control. Additionally, the raster can be variously blanked, defocussed or othewise radically altered, as for example if it is turned into a spiral sweep, and so on, all having predetermined effects on the output image and the manner in which the system will break up or distort any drawing or painting laid down by hand point-by-point, but only so long as any particular image manipulating circuit is turned on. The raster can also be distorted or wiggled or moved electronically or optically with acoustical, mechanical-optical or the like systems.

Before describing each of the manipulative functions in this embodiment of my invention it must be understood that they are activated and varied by means of individual keys on a piano keyboard 25 or typewriter-like keyboard not shown. FIG. 4 shows the keys, highlighting the detailed operation of one. The key 26 is pivoted near its rear at 27. An extension of spring metal 28 is engaged with microswitch 29 such that the slightest touch of a finger of hand 30 onto the key will activate it and switch in the appropriate circuit action. At the same time, movement of key 26 will slightly rotate cam 31, and cause it to turn potentiometer 32 by means of the gear surfaces on 31 and 33. Potentiometer 32 is the control element in the same circuit switched on by microswitch 29. Thus a person's touch will activate a circuit and finger pressure will vary its magnitude, springing back off when the finger is lifted. Symbolic representation of this system, its key, microswitch and potentiometer, are shown at 34, 35 and 36 respectively. When no keys are activated, the system operates in the normal, undistorted or unaltered raster position, as described above.

Returning to FIG. 3, the vertical deflection generator, vert def gen 23, is a sawtooth relaxation oscillator, and the horizontal deflection generator, horiz def gen 24, is a flyback-type oscillator, generating a current sawtooth in the deflection yoke of CRT 2. The linearity, height or width, and centering resistive controls of these two circuits are brought out as shown (numbers 37 through 42.) Each of these controls are shunted with six of the variable potentiometer-switch key controls on keyboard 25. The following effects are therefore introduced, and magnitude controlled, as each key is depressed:

Vertical Linearity 37: affects length of small vertical lines, altering the length as they appear in the output, as a function of where they are drawn on the paintscreen along the vertical axis.

Horizontal Linearity 40: same affect as above, only in the horizontal direction.

Vertical Size 38: distorts the vertical size of all lines or shapes drawn on the paintscreen, distorting them along the vertical axis.

Horizontal Size 44: same affect as above, only in the horizontal direction.

Vertical Centering 39: affects position of all points introduced, changing where they will appear along the vertical axis of the output screen.

Horizontal Centering 42: same effect as above, only in the horizontal direction.

In general the system shown in FIG. 3 enables a person to play one hand over the keyboard 25 and manipulate dot-by-dot, or point-by-point, as he draws or paints with his free hand. While the particular manipulations are limited, the potential exists for introducing very comprehensive alterations in the points as they are introduced. It is obvious that a systematic organization of potential manipulations must be made, perhaps under the control of a plug-board type switching system for various different types of image-handling problems.

This system can also cause non-linear coordinate transformations between differing geomemtries of sweep. If, for example, the raster 1 is shifted completely from a rectangular coordinate system to a spiral sweep, and the output monitor maintains a conventional sweep scan, very drastic alterations will occur in all the geometries hand-drawn on the paintscreen, as viewed in the output monitor. In a spiral scan system, the geometric relationships of the scans of raster 1 and CRT 16 raster, must be such that they begin to scan their respective areas from a given point, generating lines and circles with a consistent relationship, and terminate in synchronism at another given point. Both rasters must be initiated in synchronism, and as the spiral raster is traced out, by a deflection system conventional to the radar art, it must turn one loop of its circle per line of the output raster, terminating at the center when the rectangular raster terminates at its end. All drawn images will therefore be transformed into a distorted, circular image, immediately visible as it is drawn, In another arrangement the invention can be used to manifest visually the discontinuities that occur when complex variables are graphed. If, for example, the paintscreen raster 1 is overlapped by reflecting it off a broken mirror before it is focussed from 3 to the wedge 4, introducing a discontinuity along a discrete line in the original paintscreen area, when a person draws across this line domain as reflected across the paint surface, the discontinuity will occur as a jump in the output at the same line domain. Discontinuities can also be introduced into the raster 1 by means of a computer, automatically predistorting it according to any given complex equation, such that any hand-drawn image geometry will be visualized immediately on the output screen in the act of the drawing. The invention therefore becomes a potential adjunct for a mathematician desiring an instantaneous view of transformations of given hand-drawn geometries or shapes transformed from one coordinate system to another.

Paintpulse Gating Manipulation

In FIG. 2, the gates are turned on in time corresponding to points within the raster 1 timing, according to where stylus or brush 8 is positioned, generating the paintpulse. If the paintpulse is used to activate signals coming from other sources, those signals will be introduced into the output screen at these points. The desired signals can be derived from a series of small vidicons, whose sweeps are synchronized by the gating paintpulse, trained on a series of specific symbols or images that a person might desire to introduce into the visual output, each switched in by additional keys on keyboard 25. Thus another large body of images can be brought into play with the keyboard, placed down anywhere within the visual output by means of the stylus or brush 8 touching the paintscreen the moment that a particular key is depressed.

The above inputs provide another large group of image manipulations which, in combination with the raster ground manipulation described above, makes the invention a versatile device for graphic creation. The multi-colored images created are all accumulated withint he storage tubes 18, 19 and 20, unless overdrawn with another images, or erased in total by turning off the storage tubes.

Music can be introduced by means of coupling a piano to the switching system of keyboard 25, or the switches of keyboard 25 can control the elements of a music synthesizer or electric organ.

Many variations are also possible, as for example if the original video derived from the video amp 7 is used to influence the raster 1, for example by means of deflection modulating it slightly, or by means of intensity modulating the beam of CRT 2. Thus all images will be distorted according to where the brush or stylus is located, much as if it were a magnet influencing the beam directly. In addition, the image of a given scene or drawing can be derived from a vidicon, applied to the grid of CRT 1, intensity modulating this scene on the raster 1, and with the above feedback, this scene or drawing will be distorted as the person places the brush or stylus over corresponding points, and manipulates them too if desired, all visible on the monitor output CRT 16, and preserved in the storage tubes or not, as desired.

Another variation introduces paintpulses electronically, eliminating the prism flying-spot painting capability, these pulses being under control of keys on keyboard 25. In such a system single pulses are generated at the raster 1 frame rate in multivibrator circuits, delayed from each other so as not to introduce coincident pulses, each turned on by keys of keyboard 25. When a person plays the keyboard, and also plays circuits that activate manipulation, he can cause images to come and go at various spots across the screen, varied as desired according to the repertory of symbols or images introduced by means of the small vidicons in the system described above.

In another variation musical switching signals, derived from a source of music whose frequencies are filtered and connected to trigger switching voltages as a function of audio frequencies, as in a standard multi-channel color organ, are used to turn on the 10 pulses generated above, in a predetermined order across the screen, also switching in designed images from vidicons as described above, according to a random or prearranged musical scheme. Musical audio frequencies would therefore cause images to appear over the visual area of the output screen as a function of the music itself, creating designs and color variations over a color television screen of a pleasing and interesting nature.

Another important variation of my invention replaces the television output with a microscopic laser or electron scanning system for the purpose of scoring, burning, etching or brasing or otherwise influencing areas or objects down to microscopic dimensions. The paintpulse in this etching system for microscopic dimensions is generated by the means outlined above, but it also can be derived from a system identical to that of the Harper invention mentioned above, which is a video-drawing system; or as described in the Hendrick patent, which is a telautographic system.

When the invention output is connected to a scanning electron microscope capable of generating a beam that can etch, score or burn microscope capable of generating a beam that can etch, score or burn miscroscopic objects, similar for example to that described by Braers and Hatzakis in "Microcircuits by Electron Beam," in the November 1972 issue of Scientific American, the paintpulse output from clipper 11 controls the beam by deflecting it on and off the axis of the electron microscope column with electrostatic deflection plates. The scanning rate of the microscope is synced to the sync generator 22, and blanking is derived from the same source. Normally a less intense electron beam is made to impinge on the object by having only a portion of the beam in the column line from a small dc level voltage, sufficient to provide enough secondary electrons from the conventional pickup systems of scanning electron microscopes to view the object. But when the paintpulse is activated it allows the full force of the electron beam to be aimed down the column to etch the object. This action can be activated for as many frame scans as desired to achieve the desired influence upon the microscopic point desired. Using this system, or the laser microscopic system described below, it is possible to alter or influence extremely microscopic dimensions for the purpose of performing delicate manual controls, or making alterations of objects even down to cellular or molecular levels, all under manual control and only limited by the resolution of the microscopic system used. While the scanning electron system is extremely delicate, a laser scanning system could do less diminutive work. In the laser beam output imaging system the laser beam is deflected across a large surface, or through a microscope, to a small area, by means of electrooptical or acoustical-optical deflectors and rotating mirrors, conventional to the laser art. See for example U.S. Pat. No. 3,488,102 by Buck et al., or U.S. Pat. No. 3,492,596 by Vorie, plus their cited references. Normally, in this application, the laser beam is blocked by a modulating cell to which the video signal containing the potential paintpulses from clipper 11 are applied. The modulator is a Pockels Cell, Kerr Cell or the like, also conventional to the laser art, that is capable of responding to pulse signal inputs. The surface or microscopic object to be influenced by this laser beam is also televised by conventional techniques for observation purposes. When a person places the hand-held stylus on the paint surface, it sends a signal through the amplifier and clipper 11 to operate the modulator, momentarily allowing the laser beam to send its full force against the hand-chosen point on the object or surface to be etched or burned.

While these systems and various methods for drawing point-by-point, and painting into color television, and manipulating the points and images created, and introduce various other types of designed images into a color output, or down to a microscopic dimension for purposes of etching or burning, represent a complete operational system, it should be apparent to anyone familiar with the art that the various elements and steps may be changed or altered, or completely supplanted by the use or substitution of other elements or arrangements of the components without departing from the spirit and scope of my invention. Accordingly, my invention should be considered to include any or all such modifications, variations and alternative forms that fall within the scope of the appended claims.