[0001] This application is a continuation-in-part of co-pending application Ser. No. 09/595,003 titled “System and Method for Automatically Generating a Graphical Program to Implement a Prototype” filed Jun. 13, 2000, whose inventors are Nicolas Vazquez, Jeffrey L. Kodosky, Ram Kudukoli, Kevin L. Schultz, Dinesh Nair and Christophe Caltagirone which is hereby incorporated by reference.
[0002] The present invention relates to the field of computer programming, and more particularly to a system and method for automatically generating a program based on program information, wherein the automatically generated program includes a graphical user interface with user interface controls for specifying or viewing input or output parameter values to or from the program. One embodiment of the invention relates to the field of computer program prototyping, and more particularly, to the automatic generation of a computer program from a prototype.
[0003] As software tools have matured and grown in popularity and complexity, it has become increasingly common for users to use software applications other than traditional application development environments to create a problem solution or design or model a computational process. For example, applications such as prototyping environments and test executives provide specialized environments enabling users to work at a higher level than a programming language does. However, there is often a tradeoff between ease-of-use and the ability to create customized solutions to a problem. For example, a user may desire the ability to enhance a solution with features that are not provided by the high level application used to create the solution. Also, it may be desirable to export the solution into other environments external to the application. Thus, in many cases it would be desirable to automatically generate a computer program which implements the solution. This would allow the user to customize the solution and integrate the solution with other applications or processes.
[0004] There are many examples of tools and applications from which it would be desirable to automatically generate a computer program. Generally, these tools and applications create information, e.g., in response to user input. When this information is used in automatically generating a computer program, the information may be referred to as, “program information”. The program information may comprise any type of information specifying functionality for the generated program.
[0005] One example of a tool which creates program information that can be used in automatically generating a program is referred to as a prototyping environment. A prototyping environment may be a specialized application for developing solutions to problems in a particular problem domain. The specialized application can provide an environment that is conducive to rapidly and conveniently prototyping a problem solution.
[0006] A prototyping environment may integrate various capabilities in order to aid developers of problem solutions, depending on the particular problem domain. For example, a prototyping environment may provide a library of operations that are specific to a problem domain and may enable a user to select and execute various operations from the library. The prototyping environment may include a graphical user interface that is streamlined for interactively experimenting with various parameters associated with the selected operations and seeing the effects of the adjusted parameters. A prototyping environment may also include capabilities for simulating real-world objects or processes. A prototyping environment may be used to create program information such as a recipe, solution, or script, also called a prototype, which represents an algorithm designed by the user in the prototyping environment.
[0007] Prototyping environments may be developed for many different problem domains. For example, a prototyping environment for image processing may enable a user to load or acquire an image and apply various image processing operations to the image, such as filtering operations, morphology operations, edge detection operations, etc. Such a prototyping environment may enable the user to build a script including various operations that are applied to images, e.g., for use in a machine vision, pattern matching, shape matching, or other imaging application. Other examples of prototyping environments include:
[0008] a sound processing environment for applying various audio operations to a sound clip, e.g., in order to analyze the sound clip, eliminate background noise, etc.
[0009] an instrumentation environment for interacting with hardware instruments, e.g., in order to initialize an instrument, acquire data from the instrument, analyze the acquired data, etc.
[0010] a circuit design environment for developing and testing circuit designs, e.g., for programmable logic devices
[0011] As described above, it may be desirable to automatically generate a computer program based on a prototype, e.g., to implement an algorithm or process specified by the prototype. In this case the program information comprises prototype information created by the prototyping environment.
[0012] Another example of a tool which may generate program information that can be used in automatically generating a program is referred to as a test executive. A test executive application allows users to organize and execute sequences of reusable test modules, e.g., to control a test involving one or more instruments. The test executive software operates as the control center for the automated test system. More specifically, the test executive software allows the user to create, configure, and/or control test sequence execution for various test applications, such as production and manufacturing test applications. Text executive software typically includes various features, such as test sequencing based on pass/fail results, logging of test results, and report generation, among others. Similarly as described above, in some cases it may be desirable to move the execution of the test sequence from under the control of the test executive software to an independent computer program. Thus, the test sequence information may be used as program information in automatically generating the program.
[0013] In other examples, program information may be created by any of various other types of tools or applications. Further examples of program information on which an automatically generated program may be based include: a state diagram; and an existing computer program (e.g., a program to be translated to a new language or environment).
[0014] In different cases, it may be desirable to automatically generate programs in various programming languages. For example, for the image and sound processing prototyping environment examples given above it may be desirable to generate programs in C/C++, Java, Basic, Fortran, or other high-level text-based programming languages, while for the circuit design prototyping environment it may be desirable to generate a program in VHDL or Verilog.
[0015] It may also be desirable to generate programs in a graphical programming language. Generating a graphical program may be especially useful for those users who are not familiar with traditional text-based programming languages, such as C/C++, Java, etc. U.S. Pat. Nos. 4,901,221; 4,914,568; 5,291,587; 5,301,301; and 5,301,336; among others, to Kodosky et al disclose one embodiment of a graphical programming environment which enables a user to easily and intuitively model a process. The method disclosed in Kodosky et al allows a user to construct a block diagram using an editor. The block diagram may include interconnected graphical icons such that the block diagram created graphically displays a procedure or method for accomplishing a certain result. In response to the user constructing the block diagram, data structures or executable code that corresponds to the displayed procedure is automatically created. This graphical programming environment may be used for creating virtual instrumentation systems, industrial automation systems, modeling processes, and simulation, as well as for any type of general programming. Graphical programming environments such as the National Instruments LabVIEW product have become very popular. In particular, graphical programming tools are being used for test and measurement, data acquisition, process control, man machine interface (MMI), supervisory control and data acquisition (SCADA) applications, simulation, and machine vision applications, among others.
[0016] In many cases, when automatically generating a program based on program information, it would be desirable to enable the program to receive input and/or provide output. For example, in the case of a prototyping environment, various functions comprised in the prototype may have associated input and/or output parameters. The prototyping environment would typically have its own graphical user interface giving users the ability to easily adjust input/output parameters for prototype functions. It would be desirable to preserve this ability for an independent program. Thus it would be desirable to automatically create a graphical user interface (GUI) for the generated program, such that during program execution, users can specify input parameter values via user interface controls in the GUI and/or may view output parameter values via user interface controls.
[0017] One embodiment of the present invention comprises a system and method for automatically (i.e., programmatically) generating a computer program based on program information, wherein the program includes a graphical user interface (GUI) for specifying input values to or viewing output values from the program. In various embodiments, the program information may comprise any type of information specifying functionality for the generated program. The program information may comprise information created by a software application. Examples of program information include: prototype information (e.g., created by a prototyping environment), test executive information (e.g., created by a test executive application), state diagram information (e.g., created by a state diagram editor), and an existing computer program (e.g., a program to be translated to another programming language or environment).
[0018] In one embodiment, the program information may comprise information specifying a prototype, and the system and method may automatically (i.e., programmatically) generate a computer program from the prototype, wherein the program includes a graphical user interface for specifying input parameter values to or viewing output parameter values from the program. By utilizing an application specialized for developing solutions to problems in a particular domain, i.e., a prototyping environment, a user may develop a solution to a problem, i.e., a prototype. The prototype may comprise a series of functional operations that form the problem solution. For example, an image processing prototyping environment may be utilized to create a series of image processing operations, e.g., to select a series or set of machine vision or image processing tools, that may be applied to an image in order to analyze the image, e.g., in order to count the number of objects of a certain shape which appear in the image.
[0019] For each functional operation of the prototype, one or more parameters may be associated with the operation. The associated parameters may include input and/or output parameters. An input parameter provides a value affecting the behavior of the operation, while an output parameter value is set by the operation. For each input parameter, the prototyping environment may enable the user to adjust the parameter value as desired, e.g., in order to experimentally see the effect of varying the parameter value. For each output parameter, the prototyping environment may be operable to display the output parameter value, once the operation has set the value. The prototyping environment may provide a graphical user interface for setting input parameter values and viewing output parameter values. As each operation is added to the prototype, input parameter values that the user has set for the operation may be associated with the operation. Parameters may have an associated data type, e.g., a simple type such as string, integer, floating point, Boolean, etc., or a complex type which is an aggregate of one or more simple types.
[0020] Once the user has developed the desired prototype, the user may request the prototyping environment to automatically (programmatically) generate a program implementing the prototype, e.g., in order to run the program independently of the prototyping environment. In various embodiments, the prototyping environment may be operable to generate programs in any of various programming languages, including text-based or graphical programming languages. The generated program may also be targeted toward a particular application development environment, e.g., to utilize proprietary features or to create files that are formatted in a manner expected by the application development environment.
[0021] In one embodiment, the prototyping environment is operable to display a list of parameters associated with the operations included in the prototype, in response to the user requesting the program to be generated. The user may select from the displayed list the input parameters which are desired to be interactively changeable. In response, the prototyping environment is operable to programmatically generate a graphical user interface for the program, wherein the graphical user interface may include a user interface control associated with each selected input parameter, or associated with groups of input parameters. Similarly, the user may select from the displayed list the output parameters which are desired to be interactively viewable, and the prototyping environment is operable to include a user interface control (in this case an indicator or display element) for viewing each selected output parameter value on the graphical user interface. The user interface control for a given input parameter is operable to receive user input, such as during program execution, and provide this input to the associated operation in the program. Similarly, the user interface control for a given output parameter is operable to receive output data from the program, such as during program execution, and display this output to the user.
[0022] It is noted that the displayed list of parameters may group one or more related parameter values into a single item. The user interface control associated with this item may include multiple portions or fields for setting each value. Thus, the term “user interface control” may refer to an aggregate of simpler user interface controls or user interface elements which together form a suitable means for receiving an input parameter value or displaying an output parameter value. As noted above, parameters may have associated data types, and these data types may be used to determine an appropriate user interface control to create for a given parameter.
[0023] Once the user has selected the desired parameters, the prototyping environment may programmatically generate the program and the graphical user interface (or may pass the prototype information to another program that performs the generation). For each input parameter that was selected, the portion of the program that receives the input parameter is generated so as to be operable to obtain the parameter value from the corresponding user interface control in the graphical user interface. Similarly, for each output parameter that was selected, the portion of the program that produces the output parameter is generated so as to be operable to set the value of the corresponding user interface control, in order to display the value in the graphical user interface. For each input parameter that is not selected by the user, the prototyping environment may hardcode a value for the parameter into the generated program. As noted above, when an operation is added to the prototype as the prototype is being created, input parameter values that the user has set for the operation may be associated with the operation. These values may be hardcoded into the program. In one embodiment, a selectable list of parameters is not displayed, and a user interface control for each parameter is automatically created, i.e., no parameter values are hardcoded into the program.
[0024] Thus, one embodiment of the present invention provides an improved system and method for automatically or programmatically generating a computer program from a prototype, wherein the program includes an improved graphical user interface. The graphical user interface of the automatically generated computer program may be used for specifying input parameter values to or viewing output parameter values from the program.
[0025] As noted above, in other embodiments, the program information upon which the programmatically generated program is based may comprise other types of information besides prototype information. Techniques similar to those described above may be used in programmatically generating a program based on the program information, such that the program includes a graphical user interface with user interface controls for specifying input values to or viewing output values from the program. For example, in a case where the program information specifies a test executive sequence, various steps in the test executive sequence may have associated parameters. Thus, when the program is generated, the user may be prompted to choose one or more of these parameters for which to generate user interface controls, similarly as described above.
[0026] In the examples given above, the user interface controls included in the program's GUI correspond in some respect to the program information used to generate the program. For example, in the prototyping environment example discussed above, the user interface controls included in the GUI correspond to parameters of functional operations specified by the prototype information. In other embodiments, an automatically generated program may include user interface controls in the program's GUI that do not strictly correspond to program information. For example, referring again to the test executive example, in addition to including user interface controls in the GUI for receiving/displaying values of parameters associated with steps of the test executive sequence, other user interface controls for other types of purposes may also be included in the generated GUI. For example, if desired, the GUI for the program may include user interface controls for turning result logging on or off, starting or stopping the test, etc. Thus, these user interface controls may be associated with additional functionality beyond what is specified by the program information itself. Thus, in various embodiments, when the user requests that the program be automatically generated, the user may be prompted for whether to include user interface controls in the GUI for any of various types of purposes (or the user interface controls may be included by default), and the automatically generated source code for the program may include code for implementing additional functionality associated with these user interface controls.
[0027] A better understanding of the present invention can be obtained when the following detailed description of the preferred embodiment is considered in conjunction with the following drawings, in which:
[0028]
[0029]
[0030]
[0031]
[0032]
[0033]
[0034]
[0035]
[0036]
[0037]
[0038]
[0039]
[0040]
[0041]
[0042]
[0043]
[0044] While the invention is susceptible to various modifications and alternative forms, specific embodiments are shown by way of example in the drawings and are herein described in detail. It should be understood however, that drawings and detailed description thereto are not intended to limit the invention to the particular form disclosed, but on the contrary the invention is to cover all modifications, equivalents and alternative following within the spirit and scope of the present invention as defined by the appended claims.
[0045] Incorporation by Reference
[0046] The following references are hereby incorporated by reference in their entirety as though fully and completely set forth herein.
[0047] U.S. Pat. No. 5,481,741 titled “Method and Apparatus for Providing Attribute Nodes in a Graphical Data Flow Environment,” issued on Jan. 2, 1996.
[0048] U.S. patent application Ser. No. 08/810,079 titled “System and Method for Developing Automation Clients Using a Graphical Data Flow Program” filed Mar. 4, 1997.
[0049] U.S. patent application Ser. No. 09/518,492 titled “System and Method for Programmatically Creating a Graphical Program” filed Mar. 3, 2000.
[0050] U.S. patent application Ser. No. 09/595,003 titled “System and Method for Automatically Generating a Graphical Program to Implement a Prototype” filed Jun. 13, 2000.
[0051] U.S. patent application Ser. No. 09/587,682 titled “System and Method for Automatically Generating a Graphical Program to Perform an Image Processing Algorithm” filed Jun. 5, 2000.
[0052]
[0053]
[0054] In various embodiments, the computer system
[0055] In one embodiment, the computer system
[0056] As used herein, the term “prototype” is intended to include solutions, recipes, scripts or other mechanisms which represent an algorithm or solution to a problem. For example, a prototype may comprise a series of functional operations that form a problem solution or algorithm for a particular problem. A prototype may also comprise an “object-oriented” or other type of solution representation which indicates functional operations that may be performed to accomplish a desired algorithm or solve a particular problem. As described below, a prototype may also include one or more input and/or output parameters which may be associated with operations in the prototype.
[0057] In one embodiment, a prototyping environment may be an image processing prototyping environment, but it may also be any of various other types of prototyping environments. For example, the prototyping environment may be used to develop prototypes or algorithms in any of various fields or disciplines, such as machine vision, image processing, robotics, process control, automation, test and measurement, simulation, audio, video, graphics, and workflow processes, among others. Examples of the prototyping environment include IMAQ Vision Builder from National Instruments Corporation, Checkpoint from Cognex Corporation, and IPLab Spectrum from Scanalytic Corporation, among others.
[0058] In various embodiments, the PG program which generates the program may also create the program information on which the program is based. For example, in one embodiment, the PG program may include or be associated with a prototyping environment application such as described above. In other embodiments, the PG program may interface with a separate program or application to receive the program information.
[0059] In one embodiment, the PG program may be implemented as a self-contained program or application that includes all necessary program logic for generating the program. In another embodiment, the PG program may comprise a client portion and a server portion (or client program and server program), wherein the client portion may request or direct the server portion to generate the program. For example, the client portion may utilize an application programming interface (API) provided by the server portion in order to generate the program. In other words, the client portion may perform calls to the API provided by the server portion, and the server portion may execute functions or routines bound to these calls to generate the program. In one embodiment, the server portion may be an instance of a graphical programming environment application. For example, the LabVIEW graphical programming environment application enables client programs to interface with a LabVIEW server in order to programmatically generate or modify graphical programs.
[0060] As used herein, the term “PG program” is intended to include any of various implementations of a program (or programs) that are executable to programmatically generate a program based on received program information. For example, the term “PG program” is intended to include an embodiment in which the PG program is a self-contained program or application (not implemented as a client/server program) that includes all necessary program logic for programmatically generating a program. The term “PG program” is also intended to include an embodiment in which a combination of a client portion (or client program) and server portion (or server program) operate together to programmatically generate the graphical program. The term “PG program” is also intended to include other program implementations.
[0061] In an embodiment in which a client program interfaces with a server program to generate the program, the server program may execute on the same computer system as the client program or may execute on a different computer system, e.g., a different computer system connected via a network. For example, in
[0062] It is noted that the PG program may be implemented using any of various programming technologies or methodologies. Where the PG program is implemented as client and server programs, each of these programs may utilize procedure-based techniques, component-based techniques, and/or object-oriented techniques, among others. The programs may be written using any combination of text-based or graphical programming languages. Also, the programs may be written using distributed modules or components so that each program may reside on any combination of computer system
[0063] FIGS.
[0064]
[0065]
[0066] The one or more instruments may include a GPIB instrument
[0067] The GPIB instrument
[0068] The GPIB card
[0069] The VXI chassis or instrument
[0070] A serial instrument (not shown) may also be coupled to the computer
[0071] The instruments are coupled to the unit under test (UUT) or process
[0072]
[0073] The one or more devices may include a data acquisition board
[0074] The DAQ card
[0075] Referring again to
[0076] In addition, the memory medium may be located in a first computer in which the programs are executed, or may be located in a second different computer which connects to the first computer over a network, such as the Internet. In the latter instance, the second computer may provide the program instructions to the first computer for execution. Also, the computer system
[0077] In one embodiment, the PG program and/or the resulting program that is automatically created are designed for data acquisition/generation, analysis and/or display, machine vision, or for controlling or modeling instrumentation or industrial automation hardware. For example, in one embodiment, the program that is generated is a National Instruments LabVIEW graphical program or a National Instruments LabWindows/CVI C program. The LabVIEW and LabWindows/CVI programming environments provide specialized support for developers of instrumentation and industrial automation applications.
[0078] However, it is noted that the present invention can be used for a plethora of applications and is not limited to machine vision, instrumentation, or industrial automation applications. In other words,
[0079]
[0080]
[0081] The computer
[0082] The main memory
[0083] The host bus
[0084]
[0085]
[0086] In step
[0087] In step
[0088] In step
[0089]
[0090] In other embodiments, the program information upon which the programmatically generated program is based may comprise other types of information besides prototype information. Techniques similar to those described below may be used in programmatically generating a program based on the program information, such that the program includes a graphical user interface with user interface controls for specifying input values to or viewing output values from the program. For example, in a case where the program information specifies a test executive sequence, various steps in the test executive sequence may have associated parameters. Thus, when the program is generated, the user may be prompted to choose one or more of these parameters for which to generate user interface controls, similarly as described above.
[0091] In the examples given above, the user interface controls included in the program's GUI correspond in some respect to the program information used to generate the program. For example, in the prototyping environment example discussed above, the user interface controls included in the GUI correspond to parameters of functional operations specified by the prototype information. In other embodiments, an automatically generated program may include user interface controls in the program's GUI that do not strictly correspond to program information. For example, referring again to the test executive example, in addition to including user interface controls in the GUI for receiving/displaying values of parameters associated with steps of the test executive sequence, other user interface controls for other types of purposes may also be included in the generated GUI. For example, if desired, the GUI for the program may include user interface controls for turning result logging on or off, starting or stopping the test, etc. Thus, these user interface controls may be associated with additional functionality beyond what is specified by the program information itself. Thus, in various embodiments, when the program is automatically generated, the user may be prompted for whether to include user interface controls in the GUI for any of various types of purposes (or the user interface controls may be included by default), and the automatically generated source code for the program may include code for implementing additional functionality associated with these user interface controls.
[0092]
[0093]
[0094] In step
[0095] As shown in step
[0096] In step
[0097] Similarly, each particular functional operation may have associated output parameters or settings. The prototyping environment may display a list of output parameters, and the user may select for which output parameters he desires the program to display a value on the graphical user interface. In one embodiment, the prototyping environment is operable to include a user interface control corresponding to each selected output parameter on the GUI, and the generated program is operable to provide program output displaying the chosen output parameters.
[0098] The input and output parameters may be displayed in any of various ways, and may be displayed separately or together. It is noted that the displayed list of parameters may group one or more related parameter values into a single item. For example, for an image processing prototyping environment which includes an image thresholding operation, the thresholding operation may utilize input parameter values indicating the minimum and maximum pixel values to use. For these two values, the displayed list may include a single selectable item called “range” which refers to both the minimum and maximum value. The user interface control associated with this item may then include two separate portions or fields for setting each value. Thus, the term “user interface control” may refer to an aggregate of simpler user interface controls or widgets which together form a suitable means for receiving an input parameter value or displaying an output parameter value.
[0099] In step
[0100] In various embodiments, different types of programs, including text-based and graphical programs, may be created. Examples of a text-based program include programs written in a text language such as C, C++, Fortran, Basic, Pascal, Java, and Cobol, among others. A graphical program may be “written” or developed in a graphical language comprising icons which represent functions, operations, or objects. The user may create the graphical program by selecting the icons and connecting them together in a desired way. The icons may be connected in various manners, such in a data flow, execution flow, and/or control flow format. In some embodiments, a graphical program has an underlying textual program representation, and in other embodiments, a graphical program does not have an underlying textual program representation. Examples of graphical programming development environments include LabVIEW, BridgeVIEW, DasyLab, and DiaDem from National Instruments, VEE from Agilent, Sanscript from Northwoods Software, WiT from Coreco, and Vision Program Manager from PPT Vision, among others.
[0101] As described below, the program associated with the displayed user interface may interact with another program, e.g., a programming development environment application, in order to create the program implementing the image processing algorithm. Also, as described above, step
[0102] Step
[0103] As noted above,
[0104] Image Processing Prototyping Environment
[0105] One particular example of a prototyping environment is discussed below, in order to gain a better understanding of one embodiment of the invention. It is noted, however, that the techniques described herein may be applied to various other domains, as desired.
[0106] FIGS.
[0107]
[0108] In various embodiments, the image prototyping environment may be operable to load and manipulate any of various types of images, including gray-level and color images. The prototyping environment may also support complex images in which pixel values have a real part and an imaginary part. The images may be obtained from any of various sources. The images may, for example, be obtained from an image file, such as a BMP, TIFF, AIPD, JPG, or GIF file, or a file formatted according to another image format. The images may also be obtained from a hardware device, such as a camera. For example, images may be obtained from a device such as the video device
[0109] The prototyping environment may support any of various image processing functions or operations. As used herein, the term “image processing” may include functions for processing an image, such as:
[0110] filtering functions for smoothing, edge detection, convolution, etc.
[0111] morphology functions for modifying the shape of objects in an image, including erosion, dilation, opening, closing, etc.
[0112] thresholding functions for selecting ranges of pixel values in grayscale and color images
[0113] particle filtering functions to filter objects based on shape measurements
[0114] The term “image processing” may also include functions for performing various types of image analysis, including:
[0115] a histogram function that counts the total number of pixels in each grayscale value and graphs it
[0116] a line profile function that returns the grayscale values of the pixels along a line drawn through the image with a line tool and graphs the values
[0117] particle analysis functions that computes such measurements on objects in an mage as their areas and perimeters
[0118] a 3D view function that displays an image using an isometric view in which each pixel from the image source is represented as a column of pixels in the 3D view, where the pixel value corresponds to the altitude.
[0119] The term “image processing” may also include functions useful in machine vision applications, including:
[0120] an edge detection function that finds edges along a line drawn through the image with a line tool
[0121] a pattern matching function that locates regions of a grayscale image that match a predetermined template
[0122] a shape matching function that searches for the presence of a shape in a binary image and specifies the location of each matching shape
[0123] a caliper function that computes measurements such as distances, areas, and angles based on results returned from other image processing functions
[0124] a color matching function that quantifies which colors and how much of each color exist in a region of an image and uses this information to check if another image contains the same colors in the same ratio
[0125] It is noted that the image processing functions listed above are exemplary only and that, in various embodiments of the image prototyping environment, other types of image processing functions or operations may be supported.
[0126] The user interface of the prototyping environment may enable the user to load or select an image and easily select image processing functions to apply to the image. One element illustrated in
[0127] The
[0128] A “blob” (Binary Large OBject) is a connected region or grouping of pixels in an image in which all pixels have the same intensity level. In a binary image, the background is zero, and every non-zero pixel is part of a binary object. Finding the size, number, or location of blob regions in an image may be useful for many applications, such as detecting flaws on silicon wafers, detecting soldering defects on electronic boards, locating objects in motion control applications, etc.
[0129]
[0130] Based on the line profile grayscale values shown in
[0131] The next step the user may want to perform in the blob analysis is a thresholding operation to separate the particles from the background. A thresholding operation converts the grayscale image to a binary image, keeping the pixels of interest and removing the remaining pixels. The
[0132] It is noted that for each type of image processing function supported, the prototyping environment may be operable to display intermediate windows or screens that the user interacts with. These intermediate windows or screens may enable the user to specify various parameters or settings that apply to a particular image processing function. When the image processing function is added to the script, the specified parameters may be associated with the image processing function. The parameters may then be used when generating the desired program to implement the image processing algorithm, as described below. It is noted that various image processing functions may have output parameters as well as input parameters.
[0133] The process of developing an image processing algorithm may typically involve experimenting with various image processing functions and settings for the functions. However,
[0134] Continuing with the blob analysis example, after the thresholding operation is applied, the
[0135] The user may next want to apply a particle filter function which isolates and keeps the circular blobs and removes the non-circular blobs from the image. The
[0136] Once the user has developed the desired image processing algorithm in the prototyping environment, the user may test the algorithm on other images. As described above, a script describing each step of the algorithm may be recorded, and the prototyping environment may be operable to “replay” this script on other images. For example,
[0137] FIGS.
[0138] Once the user has developed and tested an image processing algorithm in the image prototyping environment, the prototyping environment (or a separate program) may be operable to automatically generate a program to perform the algorithm.
[0139] In step
[0140] As described above, in various embodiments, any of various types of programs, including text-based and graphical programs, may be generated to implement the image processing algorithm. In step
[0141]
[0142] As shown in
[0143] In step
[0144] In step
[0145] In step
[0146] In step
[0147] One or more files including program code may be generated in step
[0148] As noted above, the flowchart of
[0149] Maintaining an Association Between a Script and a Generated Program
[0150] In one embodiment, an association between the script created in the prototyping environment and the program generated from the script may be maintained. This association may enable a user to run the automatically generated program and then return to the prototyping environment in order to view or edit the script used to create the program. The ability to return to the prototyping environment may be useful, for example, if while using the program to process images, it is discovered that the program has an error or needs to be modified for other reasons.
[0151] The association between the script and the generated program may be implemented in any of various ways. For example, in one embodiment, a program file may be enabled to store information related to the program; thus, the script information may be stored in the program file when the program is created. In another embodiment, the script information may be stored in another file associated with the program, such as a project or resource file. In another embodiment, the prototyping environment may be operable to store the script information. For example, when the program is created, the prototyping environment may determine or receive a key usable for uniquely identifying the program, and this key may later be used by the prototyping environment to retrieve the script information used to create the program. In another embodiment, information specifying the script used to create the program may not be maintained, but rather, the prototyping environment may be operable to read the program and automatically re-create the script used to create the program.
[0152] In various embodiments, a user may perform any of various actions in order to return to the prototyping environment to view or edit the script associated with a program. For example, the program may be a program associated with a development environment that is coupled with the prototyping environment, and the user may be able to automatically return to the prototyping environment by issuing a command to the development environment, e.g., by selecting a menu item. In another embodiment, the user may first view the user interface for the prototyping environment and then select the script corresponding to the program.
[0153] In one embodiment, a program created from a script may be “locked”. While the program is locked, the user may be prevented from modifying the program. The ability to lock the program may be useful, for example, in order to ensure that the state of the program corresponds to the state of the script used to create the program when the user returns to the prototyping environment. If the user desires to modify the program, the user may unlock the program, e.g., by selecting an option in a development environment associated with the program. The mechanisms of locking and unlocking a program may be implemented in any of various ways, according to methods supported by a particular system or programming environment.
[0154]
[0155] As discussed above, any of various types of programs, including graphical programs, may be automatically generated from a script developed in the prototyping environment.
[0156] Step
[0157] As described above, a graphical program diagram may comprise nodes which are connected together to model the program data flow or control flow, where various nodes represent different functional blocks. For each image processing function step of a script, the step may be implemented in a graphical program using one or more nodes that are connected to each other appropriately. For example,
[0158] Thus, the information retrieved in step
[0159] The information retrieved in step
[0160] As discussed above, the user may designate certain parameter values that he wishes to be able to change or display interactively. For these parameters, a leaf node constant representing the parameter value is preferably not connected to the node input/output that receives/generates the parameter value. Instead, a user interface control or indicator representing the parameter value may be created, and the node input or output that receives or generates the parameter value may be configured to receive or set the value of this user interface control or indicator. In one embodiment, when a user interface control or indicator is created, a corresponding node may be automatically created in the graphical program diagram. Thus, the output or input of the node corresponding to the user interface control/indicator may simply be connected to the node input or output that receives or generates the parameter value. For other embodiments, the node input/ouput may be connected to the user interface control/indicator value in any of various other ways, as supported by the particular embodiment.
[0161] The parameter information obtained in step
[0162] In step