Plaque It!
Sponsored by: Flash of Genius |
[0001] 1. Technical Field of the Invention
[0002] This invention generally relates to a system and methods for digital image verification and, more particularly, to an image capturing system and methods for automatically recording and watermarking a plurality of camera and image parameters with a digital image (still and video) for verifying the authenticity of the digital images.
[0003] 2. Description of the Related Art
[0004] It is often necessary to determine the conditions and circumstances (such as time, date, and location) in connection with the capturing of an image. For instance, such information can be of immense value to insurance agencies (e.g., real-estate, auto, and fire), hospitals, news agencies and crime investigating agencies, for confirming the details surrounding an accident so as to assist in the investigation of the accident and preparing the necessary accident reports. Moreover, this information would also be useful for image search algorithms that are based on such information. Therefore, an image capturing device which can automatically record a plurality of parameters with each captured image, such as names of geographic locations, altitude, longitude, time, date, photographer identification, as well as image data such as light intensity, shutter speed and flash status, would be very useful to such agencies.
[0005] Furthermore, an image capturing system which could automatically watermark (i.e., hide) the plurality of recorded parameters into each image would be useful for verifying the authenticity of digital pictures, as well as verifying and confirming the circumstances and conditions surrounding the capturing of the digital image. In general, the ability to prove the authenticity of digital images is a burdensome task because there are various commercially available software applications which allow users to manipulate and modify digital images. Therefore, by invisibly watermarking parameters associated with a digital image within the image, the authenticity of the digital image may subsequently be verified by extracting the watermarked parameters and then comparing the extracted parameters with the initially recorded parameters to determine whether they are similar.
[0006] There are commercially available film cameras which are capable of recording the time and date on each photographic image. These conventional film cameras, however, do not record additional camera parameters such as location (e.g., altitude and longitude) in an electronically accessible form. Moreover, these traditional film cameras allow a user to set any desired date and time to be recorded with each image, which limits the use of such recorded information for purposes of authenticating digital images.
[0007] There are digital cameras that are commercially available which can capture and record the time and date with each image. But like the conventional film cameras, the user can set any desired time and date to be recorded with each image. Moreover, the time and date information are often stored in the header line for the image file. Some digital cameras store these camera parameters in a separate file. The separate file may be encrypted and transmitted along with the image file. The disadvantage with this approach, however, is that the user may create his own camera parameters for an image and then encrypt it and present it as the secure data for the image. Thus, separating the captured image from the encrypted parameter file does not afford a solution to potential tampering of such recorded information.
[0008] In addition, there are commercially available cameras which allow other parameters (in addition to time and date) to be recorded with each image. These cameras, however, do not provide means for detecting if such parameters have been modified or if the images have, otherwise, been tampered with. For instance, U.S. Pat. No. 4,574,319 to M. Konishi entitled, “
[0009] Next, U.S. Pat. No. 5,506,644 to Suzuki et al. entitled, “
[0010] Furthermore, U.S. Pat. No. 5,247,356 to J. Ciampa entitled, “
[0011] On the other hand, U.S. Pat. No. 5,499,294 to G. Friedman, entitled, “
[0012] Based on the above prior art, there are no commercially available image capturing devices which afford automatic recording and watermarking of a plurality of camera and image parameters with each captured digital image. By watermarking (i.e. hiding) the recorded parameters throughout each captured image in a non-obvious manner (in addition to, or in lieu of, storing the parameters in an image header or in a separate file), it is difficult to detect and tamper with the embedded data. Accordingly, the watermarked parameters may be recovered and checked against the parameters which were recorded (e.g., electronically or manually) at the time the image was taken so as to verify the authentic of the image as well as the recorded parameters.
[0013] The present invention is directed to an image capturing system and method for automatically watermarking a plurality of recorded parameters such as the time, data, location (e.g., latitude, longitude and altitude) of the camera into each captured image. In a preferred embodiment, the watermark (or “stamping information”) consists of a stream of binary data (i.e., bits) representing one or more of the plurality of recorded parameters. During an image stamping process, the stamping information is invisibly embedded (i.e., stamped) into a source image in accordance with a defined mapping process to produce a stamped image. The mapping process associated with each image is identified by a corresponding decoding “key” (or “verification key”). The stamping process distributes the stamping information in the entire image without generating any visible traces of the stamping information or any visual artifacts in the image, thereby making it difficult to modify the image without affecting the watermarked data. In other words, the stamping information is perceptually invisible and any modification on the image alters the stamping information, and an attacker cannot restore the altered invisible mark stamped onto the image.
[0014] During an image verification process, the stamping information (i.e., the watermarked parameters) may be decoded (i.e., extracted) with the corresponding verification key. The extracted information (i.e., camera parameters) is then compared with the camera parameters recorded at the time the corresponding digital image was generated. If the extracted parameters match the originally recorded parameters, the corresponding image is determined to have not been altered since the time the image was stamped.
[0015] In an alternate embodiment where the photographer can be trusted, the recorded camera parameters associated with the image may be stored in a separate memory block associated with the generated image (e.g., in an image header) or directly transmitted to a central server.
[0016] In one aspect of the present invention, an image capturing system for automatically recording and watermarking a plurality of parameters in a captured image comprises:
[0017] a central processing unit for controlling a plurality of functions and operations of the system;
[0018] image capture means, operatively connected to the central processing unit, for generating a digital image of an observed image frame and for generating a plurality of image data associated with the generation of the image;
[0019] wireless communication means, operatively connected to the central processing unit, for receiving data from objects in the observed image frame when the image is generated;
[0020] geographic location determining means, operatively connected to the central processing unit, for determining geographic coordinates of the system when the digital image is generated;
[0021] means for determining a time and a date when the image is generated;
[0022] information receiving means, operatively coupled to the central processing unit, for receiving data associated with a user of the system when the digital image is generated;
[0023] image processing means for receiving the plurality of parameters and recording the plurality of parameters with the generated digital image, the plurality of parameters including the plurality of image data, the object data, the time data, the date data, the location data, and the user data; and
[0024] means, operatively coupled to the image processing means, for watermarking the plurality of parameters into said image.
[0025] These and other objects, features and advantages of the present invention will become apparent from the following detailed description of illustrative embodiments, which is to be read in connection with the accompanying drawings.
[0026]
[0027]
[0028]
[0029]
[0030]
[0031]
[0032] Referring to
[0033] The digital camera
[0034] The camera
[0035] In addition, a flux gate magnetometer (FGM)
[0036] The camera
[0037] A Personal Area Network (PAN) receiver
[0038] The camera
[0039] These parameters (in addition to other parameters discussed below) are generated with each digital image and provided to the image/parameter processor
[0040] In addition, the user-specified parameters may be chosen (or precluded) via voice activated commands (e.g., by stating into the microphone
[0041] Referring to
[0042] The camera
[0043] The following table illustrates the parameters that can be recorded and watermarked with each image. It is to be understood, however, that the table is not illustrative of every parameter which may be recorded and watermarked and that one of ordinary skill in the art could envision additional parameters that may be recorded and utilized in accordance with the teachings of the present invention herein.
Length Field Description string (Bytes) Location Latitude (e.g. N 41 45.721), Longitude 72 (e.g. W 76 35.420) Altitude (e.g., 1.500E + 2 m) Principal axis Azimuth (e.g., 030.12 degrees), 12 orientation Altitude (+20.19 degrees) Camera Velocity in m/sec or ft/sec (e.g., 1.100E + 2 8 m/sec) Image mode Portrait (P) or Landscape (L) 2 Image quality high, medium, low or other 1 Compression scheme e.g., JPEG 1 Date day/month/year (e.g., 12/12/1997) 10 UTC Time hr:min:sec (e.g., 10:20:31) 8 Local Time hr:min:sec (e.g., 10:20:31) 8 Exposure Duration hr:min:sec (e.g., 1.000E + 1 msec) 8 Aperture f-stop (e.g., f11) 2 Light meter reading lux (e.g., 1.001E + 0 lux) 8 Flash Status ON/OFF 1 Lens focal length in mm (e.g., 5.000E + 2 mm) 8 Auto focus distance in m (e.g., 1.510E + 2 m) 8 Frame no frame counter value since last reset 4 Voice annotation ascii text 20 Photographer ascii text 30 211
[0044] As shown in the table, the memory required for storing the recorded parameters is a few hundred bytes, which is significantly less than the memory needed to store a compressed image. For instance, a 1152×864 digital image captured by the commercially available Kodak DC
[0045] The preferred watermarking process in accordance with the present invention (as disclosed in the above U.S. application Ser. No. 08/780,484) will now be described in detail with reference to
[0046] A verification key
[0047] The stamped images are stored in an image archive
[0048] It is to be appreciated that image verification processing can also be performed with the watermarker processor
[0049] Referring now to
[0050] Once the desired parameters are watermarked, the stamped image and corresponding verification key will then be stored within the camera (step
[0051] When the authenticity of a given image must be verified, the associated stamped image will be downloaded from memory (step
[0052] Preferred Image Stamping and Verification Methods
[0053] Referring to
[0054] The functional block
[0055] As discussed in the above incorporated U.S. application Ser. No. 08/780/484, the verification key can be a set of binary look-up-tables (LUT's), or can be a number and a known function generator of the number which can be used to generate the required set of binary LUT's. To produce the key, a random number generator
[0056] The binary LUT's serve as the binary mapping functions in the computation of watermark values of the source image pixel
[0057] EW(I,J)=LUT
[0058] The computed watermark value of a source image pixel is then compared with the desired value of the stamping information W(I,J), in the comparison process as shown in block
[0059] The gateway
[0060] For a color image pixel, the three color values of the pixel are altered. The pixel values have to be modified in a way such that the distortions are minimal. Any color intensity value can be modified at a given iteration. The level of adjustment starts from +1 or −1 and gradually increases in magnitude in subsequent iterations. Book-keeping of the current level of adjustments in each color component is used to ensure that the magnitude of level change increases gradually. The maximum level of adjustment is controlled to ensure the alteration does not lead to significant difference in visual quality of the image.
[0061] The selection of color to be altered at a given iteration can be achieved by using predefined tables or by incorporating random number generators in the selection process. The advantage of using random number generators is that the modifications are spread out to all the three colors instead of concentrating at a particular color for a majority of the pixels. The effect is especially prominent in the encoding of synthetic images in which a large area may be of uniform intensity values. A systematic scheme in the selection of pixels to modify can lead to the concentration of alteration in one particular color (+or −) of level adjustment is also made random at each pixel to smooth out the errors produced locally.
[0062] For a monochrome image, the pixel alternation is a straight forward adaptation of the method used in the alternation of color pixels: the gradual adjustments of the pixel values are confined to one intensity component instead of the three color components.
[0063] The errors produced by the pixel value alteration described above are then diffused as shown in block
[0064] Error diffusion is commonly used for smoothing the transitions of colors around neighboring pixels in the conversion of true colors to limited color palettes. The process produces good quality color images by adding natural texture and color transitions in place of sudden changes. Error diffusion is employed for similar reasons in the encoding unit such that the errors produced by altering the pixel values (which is equivalent to the quantization errors in the color conversion process) will spread to neighboring pixels. This method enables retention of the proper average color and eliminates unwanted local artifacts. Specifically, as discussed in detail in the above incorporated U.S. application Ser. No. 08/780,484, the process of embedding stamping information
[0065] When the stamping process is completed, a stamped image
[0066] Referring now to
[0067] Referring now to
[0068] The watermark extraction function WX(*) is then applied to the pixel SS(I,J) to produce the extracted stamping information at the pixel EW(I,J) (step
[0069] The pixel indices are then incremented (step
[0070] For a stamped color image SS(I,J), every pixel (I,J) in the image contains three color-component values showing the red, green and blue color intensities respectively, namely SS
[0071] where WX(*) can be a mapping function which can take an input value (that is, the pixel intensity value) into a binary output value, and the mapping function is given by the verification key.
[0072] A different mapping function may be applied for each color intensity value of a pixel, and each mapping function for a single color component is called a color-component mapping function. For a color image, a set of three color-component mapping functions are needed, namely F
[0073] These different mapping functions are implemented in the form of binary look-up-tables (LUT's) in which the pixel value serves as the index to a table entry, and the table entry then gives a one-bit value (‘0’ or ‘1’) output. In this case the verification key is related to a set of binary look-up-tables (LUTs) (three tables for color image and one table for monochrome image), and for the exemplary embodiment may be a number (or a “seed”) which can be fed into a known cyclic function generator to generate the set of binary LUT's.
[0074] Each binary LUT of the present exemplary embodiment is generated as follows. First, given an index, the table entry is either a ‘1’ or an ‘0’, which is determined in a random fashion. For example, a pseudo random number generator can be used to generate the table entries as arbitrary numbers having a random number distribution. As is known in the art, these pseudo random number generators employ a cyclic functions that receives an input number, called a “seed,” and then recursively generate a random output value. In one embodiment, the function subroutines srand( ) and rand( ) of the standard library of the C programming language are used. For each random output value, the table value is set as ‘0’ if the random output value is even, and ‘1’ if the random output value is odd. However, as would be apparent to one skilled in the art, the random table entries could be generated in any number of ways; for example, by using values generated by linear feedback shift registers, values from random number tables, or any other orthogonal function sequences.
[0075] The watermark extraction step
[0076] for a color stamped image (and EW(I,J)=LUT(SS(I,J)) for a monochrome image).
[0077] For example, if an image pixel SS* has the color intensity values
[0078] and
[0079] then the extracted watermark value EW* of SS* becomes
[0080] In the above example the watermark extraction function WX(*) is the combination of the XOR function and the mapping functions given by the LUT's as shown in
[0081] The comparison of extracted watermark values EW(I,J) and the watermark image values W(I,J) as shown in the block
[0082] Although the illustrative embodiments of the present invention have been described herein with reference to the accompanying drawings, it is to be understood that the invention is not limited to those precise embodiments, and that various other changes and modifications may be affected therein by one skilled in the art without departing from the scope or spirit of the invention. All such changes and modifications are intended to be included within the scope of the invention as defined by the appended claims.