Plaque It!
Sponsored by: Flash of Genius |
[0001] The present invention relates to the field of communication devices that include the functionality of personal digital assistants; more particularly, the present invention relates to a human-machine interface for operating a personal digital assistant having expanded capabilities including wireless communication capabilities.
[0002] The popularity and use of personal digital assistants (PDAs) and wireless telephones has increased in recent years. A PDA or hand-held computer is primarily a lightweight, compact productivity and communications tool that can typically be held in one hand, leaving the other free to input data with a pen type stylus or a reduced size keyboard. In some cases, virtual keyboards (i.e., keyboard representations displayed on a touch screen display unit) may be used for data entry. A PDA provides computing and information storage and retrieval capabilities for personal or business use. Typical uses include schedule and address book storage and retrieval, as well as note taking functions. In addition, many PDAs are capable of running a variety of application software packages (e.g., currency converters, calculators, text and/or image editors, etc.).
[0003] Wireless telephones are integrated radio transmitter-receivers that are capable of accessing a vast web of existing telephone connections. Increasingly, wireless telephones are being combined with PDAs in order to perform more advanced functions, such as transmitting, receiving and displaying text messages. However, typical PDA/wireless telephone combinations include a small display area which makes it very difficult to use such units for anything other than displaying telephone numbers and, possibly, the status of a call. Where some PDA/wireless telephone combination units have utilized larger screens, the user interfaces associated with these units still has not been well provided for. That is, there has been poor integration of the different functionalities of the device and/or the user has been left to choose between using the device as a PDA or as a wireless telephone, each with its own separate user interfaces.
[0004] A user interface may be thought of as the junction or meeting point between a user and a computer program. In general, an interface is a set of commands or menus through which a user communicates with an application program and/or a device. A command-driven interface is one in which a user enters commands directly to control program/device actions. A menu-driven interface is one in which a user selects command choices from various menus (often with easy to understand shorthand names or labels for more complex commands) displayed on a screen. Graphical user interfaces (GUIs) that utilize windows, icons and various menus have become standard equipment on personal computers and PDAs, and are starting to appear on some wireless telephones.
[0005] GUIs take advantage of a computer system's graphics capabilities to make a program or device easier to use. Well designed GUIs can free a user from having to learn complex command or even menu option sequences for performing tasks. Many GUIs utilize one or more of the following attributes:
[0006] a. Pointers. These are symbols that appear on a display screen and that a user can manipulate to select objects and commands. Points may appear in various forms (sometimes depending upon the application context), such as small arrows, I-beams (often found in text processing applications), and other forms.
[0007] b. Pointing device. This is a device that allows the user to manipulate the position and operation of the pointer. Common examples are mice, trackballs and joysticks, but pen type pointing devices are commonly used with hand held computer systems.
[0008] c. Icons. These are small pictures that represent commands, files or windows. By moving a pointer onto an icon with a pointing device and then manipulating the pointing device (e.g., by clicking the mouse or tapping the screen with the pen), a user can execute the command associated with the icon (e.g., launching an application program, opening a window, etc.). Icons can often be moved around a virtual desktop as if they were real objects, to simulate and allow a user to customize a work environment.
[0009] d. Desktop. This is usually regarded as the area on a display screen where icons are grouped.
[0010] e. Windows. These are used to divide a display screen into different areas. Within a window, a user can run an application program or display a file. Windows can often be moved and sized under user control to allow for customization of the work area. Although icons can be grouped within windows, windows differ from the desktop in that ordinarily applications do not run within/on the desktop. Windows can be cascaded or tiled to bring currently executing or operating programs to the foreground for interaction with a user.
[0011] f. Menus. Most GUIs allow a user to execute commands by selecting choices from a menu. As the name implies, these are virtual representations of lists of possible commands or options that can be selected by a user. Selection from a menu is often made by highlighting the desired option with a movable cursor and then selecting the highlighted option (e.g., by clicking the mouse button or tapping the screen with a pen device). In some cases, the highlighting/selection operation can be performed simultaneously by using a special selection action, such as a double click of a mouse or a double tap of a pen.
[0012] Menus come in several forms, such as pop-up menus (which appear temporarily when a user manipulates the pointer device and usually disappear after the user makes a selection or moves the pointer/cursor), cascading menus (in which submenus open in response to a menu selection), pull-down menus (which are generally special types of pop-up menus that appear directly beneath a selected command), moving-bar menus (in which options are highlighted by a bar that moves from one item to another under user control), menu bars (which are usually arranged horizontally across a screen and may have associated pull-down menus for the various command group options included therein), and tear-off menus (which are pop-up menus that can be moved around the screen like a window).
[0013] Because the user interfaces associated with current PDA/wireless telephone devices have to date not been satisfactory, it would be desirable to have an improved user interface for such devices.
[0014] In one embodiment, a handheld communication device has an associated a user interface that includes a main screen segregated into a number of screen areas. At least one of the screen areas is devoted to a system application for displaying system status icons for the handheld communication device, and another of the screen areas is devoted to a browser for displaying current or recent application information for respective user applications, which user applications may be accessed through interaction with the browser. The application information may be application status information, recent communication messages (such as e-mail messages, SMS messages, or instant messaging messages) to or from the handheld communication device and/or one or more upcoming events or to do items for a user of the handheld communication device. The user interaction may occur through selection with a pointing device such as a stylus, a fingertip stylus, a finger, or one or more buttons, or even through voice commands. The browser may also include an area reserved for displaying advertisements.
[0015] The status icons may be one or more of: a logo icon, a program group icon, an application name icon, a battery life indicator icon, a current time indicator icon, a received signal strength indicator icon, a help icon, a voice mail indicator icon, an input toggle icon, a backpack software icon, or a Bluetooth icon. The user applications may be one or more of: communication applications, personal information management applications, personal productivity applications, multimedia applications, or game applications. Such communication applications may include one or more of: a telephone dialer, an e-mail application, a short messaging service application, an Internet browser, a wireless access protocol browser, an instant messenger application, or a Bluetooth communication application. The personal productivity applications may include one or more of: a text editor, a spreadsheet, a database, an e-book reader, an image viewer, an image editor, or an audio manager.
[0016] Different categories of user interactions with the browser or interface page may initiate different application responses. These different categories of user interactions may include: a single selection operation, a select and hold operation and a double selection operation.
[0017] In one embodiment, the current or recent application information is navigable in a circular fashion within the recent application list within the main menu. The system application executes, in one embodiment, as a system bar in an area_located at the top of a display area of the handheld communication device. The browser then executes the primary interface in an area below the_system application area. The primary interface may consist of multiple linked pages and—include an area at the bottom of its associated pages for displaying advertisements or other information and this area may be configured to display one or more of streaming media, flash animations, banner ads, packet video, gif animations and/or other multimedia content.
[0018] Preferably, the area for the system application includes an area reserved for display of a logo icon. Selection of the logo icon causes a folder menu to be displayed on a display of the handheld communication device which lists the application groups present on the device. The folder menu includes an area reserved for a listing of recently used ones of the user applications, which may be displayed as icons within the reserved area. The folder menu may be arranged as a dropdown menu of available application groups for the handheld communication device.
[0019] Such application groups may include, but are not limited to, one or more of the following: an all application group, a communication application group, a multimedia application group, a personal productivity application group, a main application group that includes a personal information manager application, a games application group, a system application group, and a user personal applications group. Selection of one of the application groups causes a window that includes images representing those of the user applications that belong to the selected application group to be opened and displayed.
[0020] The present invention is illustrated by way of example, and not limitation, in the figures of the accompanying drawings in which:
[0021]
[0022]
[0023]
[0024]
[0025]
[0026]
[0027]
[0028]
[0029] Described herein is a user interface for a handheld communication device that includes both PDA and wireless telephone functionality. However, this is only one example of the use of the present user interface and other examples include use with PDAs alone or with wireless telephones alone. In some cases, the present user interface may also be used with personal computers and the like, or even automobiles that include built-in displays for controlling automobile, telephone or computer operations. Accordingly, the examples of use of the user interface presented below should be regarded as illustrative only and should in no way be seen as limiting the broader scope of the present invention.
[0030] Some portions of the detailed description that follows are presented in terms of algorithms and/or symbolic representations of operations on data within a computer memory (e.g., through the use of flow charts and/or descriptive text). These algorithmic descriptions and representations are the means used by those skilled in the computer science arts to most effectively convey the substance of their work to others skilled in the art. An algorithm is here, and generally, conceived to be a self-consistent sequence of steps leading to a desired result. The steps are those requiring physical manipulations of physical quantities. Usually, though not necessarily, these quantities take the form of electrical or magnetic signals capable of being stored, transferred, combined, compared and otherwise manipulated. It has proven convenient at times, principally for reasons of common usage, to refer to these signals as bits, values, elements, symbols, characters, terms, numbers or the like. It should be borne in mind, however, that all of these and similar terms are to be associated with the appropriate physical quantities and are merely convenient labels applied to these quantities. Unless specifically stated otherwise, it will be appreciated that throughout the description of the present invention, use of terms such as “processing”, “computing, “calculating”, “determining”, “displaying” or the like, refer to the actions and processes of a computer system, or similar electronic device, that manipulates and transforms data represented as physical (electronic) quantities within the computer system's registers and memories into other data similarly represented as physical quantities within the computer system's memories or registers or other such information storage, transmission or display devices.
[0031]
[0032] PDA telephone
[0033] PDA telephone
[0034] Hinges
[0035] Microphone
[0036] Display
[0037] PDA telephone
[0038] PDA telephone
[0039] Because housing
[0040] The speaker may also operate in a loudspeaker mode so as to permit speakerphone operation of PDA telephone
[0041] In addition to the above, PDA telephone
[0042]
[0043] Note that the wireless telephone engine
[0044] Wireless telephone engine
[0045] Modem
[0046] Display
[0047] Smart-card engine
[0048] Address book
[0049] A user may also retrieve entries from address book
[0050] According to a further embodiment, the user may manually select an item of an entry displayed on display
[0051] GPS engine
[0052] Synchronization circuitry
[0053] PDA telephone
[0054] Voice/speech recognition module
[0055] Optional biometric sensors
[0056] With the above understanding of the functionality of PDA telephone
[0057] In a current embodiment, PDA telephone
[0058] Thus main screen
[0059] Main screen
[0060] As indicated above, the pages which are loaded do not necessarily need to be stored on the PDA telephone
[0061] The system status information may be displayed through the use of various icons (as discussed below), while the application information may take the form of text describing application status and/or recent or upcoming event information (e.g., for the PDA applications such as calendars, to do lists, address books, etc.), received and/or transmitted messages and/or e-mails (e.g., for e-mail and/or short message service communications). As discussed below, the various applications themselves can be accessed through user interaction with the system application and/or interface running within the browser. Thus, users may launch and interact with applications via the main screen and screens associated with the main screen.
[0062] At the top of the main screen
[0063] In the default mode, if an application has an associated application menu, that application menu will appear directly below the system menu
[0064] As indicated above, the system menu
[0065] The following Table 1 describes specific system menu
[0066] For button selection, the screen cursor
TABLE 1 System Menu Features Feature/ Item Description Logo As indicated above, a company logo may be set to be present Icon when the main screen 156 is opened (and possibly when an 164 application is running). In essence, this logo is a small graphic image that is displayed in a corner of display 106, within system menu 158. In other embodiments, a user-selected image may be set to appear in this location, in order to provide personalization of the PDA telephone 100. When a program group is open, the system menu 158 may be configured to change so that the name of that group appears beside or n place of the logo or other image. Similarly, when an application is running in the foreground, the name of the application may appear beside or in place of the logo or other image. The logo or other image/name is also an icon, which means it has some associated functionality. In one embodiment, one tap or click of the logo icon 1164 displays the main page 156. Thus, if the system menu is displayed at the top of a currently running application, the user may bring up the main page 156 by simply selecting the logo icon 164 in this fashion. Alternatively, this action may cause a folder menu to be displayed. A tap/click and hold user input may have different functionality, for example, this type of selection operation may turn off the display 106, but leave the device in an operational state. The display 106 may be reactivated in response to any other user input, such as a screen tap or a button click. In some embodiments, when the main screen 156 is open and the user clicks or taps on the logo icon 164 for a second time in succession (essentially a double-tap/click selection operation), the PDA telephone 100 may be configured to display the last place in the shell that the user was at before the main screen 156 was open. For example, a user running an application in the communications folder (see below) may have opened the e-mail and telephone applications. On a first tap or selection operation of the logo icon 164, the main screen 156 is displayed. The user can then return to the communications folder by tapping a second time on the logo icon 164. In addition, a double tap/click on the logo icon 164 from the main screen 156 may cause the contents of the next folder in the folder menu (see below) to be displayed. Battery This is an icon that provides a visual indication of a battery and Life provides the user with a means for receiving information about Indica- the current battery life. Of course, PDA telephone 100 will need tor Icon to be equipped with conventional means for determining such 166 information (such as a voltage/current monitoring circuit that can be used to obtain information regarding the current battery life remaining in the unit's battery power source). Such circuitry is well known in the art and the precise configuration of such a circuit is not critical to the present invention. A single tap or click of this icon 166 may display a pop-up menu or window that includes a description of the remaining battery life, for example: “Battery Life @ X %” where X is a value between, say 10 and 100 in increments of 5. A tap/click and hold selection operation may have a different result. In one embodiment, this type of operation will cause a system power applet to be launched. This may be an applet that allows the user to vary power operating conditions of the PDA telephone 100. Of course, other operations may be launched by such a selection input and the precise application is not critical to the present invention. Current This is an active icon that displays the current time (user set- Time table) in the conventional fashion. A single tap or other selection Indica- operation may bring up a pop-up window with the current date. tor Icon Other selection operations, e.g., a tap and hold, may be used to 168 launch a time setting applet or the time in another time zone or zones. Signal This icon may change dynamically to indicate received signal Strength strength, with a roaming sign if applicable, as is commonly Indica- found on conventional wireless telephones. For example, the tor Icon number of illuminated signal strength bars may indicate a 170 stronger or weaker received signal. The signal strength indicator icon 170 may also provide a means for launching a phone dialer or other telephone related application. Thus, one type of selec- tion operation (e.g., a single tap) may bring up the application, while another type of selection operation (e.g., a tap and hold) may bring up a different application (e.g., such as an applet that allows the user to change the current telephone settings). When the telephone functionality of the unit is not activated, the signal strength icon 170 may change to display a red “cross-out” symbol. Help Resembling a question mark (or other easily identifiable sym- Icon bol), the help icon may be used to launch a help application. In 172 response to a selection operation (e.g., a single tap/click) help files pertaining to the application or file currently displayed in the foreground may be launched. In some cases, these help files will be HTML files that can be displayed in the browser. Voice As with other wireless telephones, the PDA telephone 100 may Mail be configured for on-board voicemail storage and retrieval or, in Indica- some cases, this may be a feature provided by the network ser- tor Icon vice provider. The voicemail indicator icon 174 may appear 174 whenever there is unplayed voice mail waiting for the user. In one embodiment, one tap/click on the icon accesses the user's voicemail (e.g., by initiating local playback if the messages are stored locally or by dialing a user-defined telephone number at which such messages may be retrieved), and prompts the user for a password, or automatically enters a saved, user-defined password if such a password is needed to access the stored messages. Other selection operations, such as a tap and hold, may launch an applet that allows the user to define his/her voice- mail preferences (e.g., a telephone number to call to retrieve messages, password settings (prompt or automatic), etc.). Input As indicated above, PDA telephone 100 may be configured to Toggle respond to different types of user input, such as via a keyboard Icon (real or virtual) or through handwriting recognition (e.g., as is 176 commonly used with handheld computer systems). The input toggle icon 176 may be used to indicate the currently set user input mode. Thus, the icon may toggle between having the appearance of keyboard and another input instrument (e.g., a pen or stylus) according to the current input mode. A user may select the input method by performing a selection operation (e.g., one tap/click) on the icon to display an Input Method field. Such a field may provide details regarding the current input mode. Other selection operations, for example a tap/click and hold, may toggle between input modes: keyboard or character recog- nizer (where one, say the keyboard, is the default input method). When character recognition software is used, the icon may change from a keyboard to a pen. Back- If present, selection of the backpack icon triggers the shell to pack load a status icon that links the primary application associated Soft- with the backpack. A single tap/click opens the primary back- ware pack software. Different backpack options may include a digital Icon camera package, a GPS receiver/mapping package, a biometric (not sensor package, and others. Where no backpack is used with shown PDA telephone 100, no icon is shown in the system menu 158. but may resem- ble a back- pack or similar article) Blue- The Bluetooth icon 178 provides a visual indication of the status tooth of the short-range transceiver 130. If the icon is not visible icon Bluetooth coverage is not available. If the icon is visible, 178 Bluetooth service is available and a single tap (or other selection operation) on the icon may display a pop-up description that reads: “Bluetooth is Active”. A tap and hold (or other selection operation) may launch a Bluetooth Properties page (or other application), which lists partnerships, and could include a Bluetooth On/Off mode control as well as other controls and properties related to this service.
[0067] As described above, the main screen
[0068] Since the page displayed by the browser in the application window
[0069] The interface page can be dynamically updated either by the user, or by an automatic updating application running on the device, or by remote control initiated by a secure connection to a properly privileged host.
[0070] The user should also be able to set the order in which the various applications are listed on the main screen
[0071] The date and time displayed at the top of the main screen
TABLE 2 Main Screen Components Compo- nent Description Main Appearing below the system menu 158, this top section of the Screen application window 159, which could be used to display the date Banner (as shown), or other information appears as a banner screen area. 182 In other embodiments, this screen area is used for one of the application information displays and the date is accessible through a selection operation of the time icon 168. Con- In one embodiment, in the default state the contacts screen area tacts 184 may be used to display the device owner's name or other Area information (e.g., the PDA telephone unit's own telephone 184 number). Taping/clicking on this area 184 brings up the contacts or address book application of the PDA. A tap/click and hold may bring up the owner's personal information (which may be stored through a registration applet or other means). Phone This screen area may, in a default state, display the telephone Area status (ON or OFF) of the PDA telephone 100 and, if so desired, 186 the service provider's network name or other information. A roaming icon may also be displayed if the user is roaming out- side of his/her home service area. When tapped, or otherwise selected, the telephone dialer application may be launched. A tap/click and hold may display a telephone settings applet. Internet This screen area may, in a default state, display the Internet Area status (online or offline) of the PDA telephone 100. By tapping 186 (or otherwise selecting) this screen area, the user may launch an Internet browser application, which operates in the conventional fashion. Tap/click and hold may display a list of favorite web- sites or other information (such as an Internet connection prefer- ence menu or list of recently visited web sites, etc.). SMS This screen area may be used to displays a list (e.g., three) of Area recently received and/or transmitted instant messages using an 188 SMS application. When this area is tapped or otherwise selected, an Inbox or other message receipt applet of the SMS application may be displayed to allow the user to read newly received messages. Tap/click and hold may launch the application and allow a user to input text for a new message. Any unread messages may be shown in BOLD font type for easy identification. Instant This screen area may be used to display the status of an instant Messag- messaging application: Online, Busy, Away from My Desk, Out ing to Lunch, On the Phone, etc. A tap/click in this area may launch Area the application to allow a user to read/respond to a message or to (not create a new message. The main screen 156 can only accommo- shown) date so many applications at one time before readability is diminished to a point that a user may find unacceptable. There- fore, a user may choose which application s/he wants to display in the main screen 156. In the present illustration, the user has decided not to display an instant messaging screen area, but additional or different screen areas may be added or deleted at will. Also, if a PDA telephone 100 is not configured with a particular application, the present user interface will not reserve main screen area for a “missing” application. That way, the entire display area may be used to display information for applications that are present. E-mail This screen area may be used to display the most recent (e.g., Area three) transmitted and/or received e-mail messages. In another 190 embodiment, only high priority messages may be displayed or user-selected messages may be displayed. When this screen area is tapped or clicked, the Inbox of the e-mail application may be displayed to allow a user to read newly arrived or previously received e-mail messages. Tap/click and hold may launch the e-mail application and allow a user to compose and send a new message. Unread messages may be displayed in BOLD font type in the e-mail screen area 190 to allow for easy identification. PIM This screen area may be used to display upcoming appointments (Person- and/or to do items and/or a contacts (address book) list. When al Infor- the area is tapped or clicked, the PIM application may be mation launched to allow a user to interact with his/her PIM (e.g., to Man- make to do items as completed, to cancel or add new appoint- age- ments, etc. Tap/click and hold may immediately take the user to ment) a new event input screen, allowing the user to quickly update Area his/her PIM with a new item and then return to the main screen 192 156 by tapping the logo icon 164. In one embodiment, the default information management view displays the next scheduled event. This screen area contains a scroll bar when scrolling is required. Footer This is a banner graphic section, which could contain an image, Area company information or dynamically loaded advertisements. 180 When tapped or clicked, the banner changes to a full screen view of whatever was shown in the footer area 180. If the banner was an advertisement the browser may open to a related Internet web site. A tap/click and hold may invoke different functionality, such as launching a settings menu that allows the user to modify the settings of the main screen 156 (e.g., the order of the screen areas, including which are displayed and which are not, and/or their contents).
[0072] As discussed above, different navigation and selection options exist for the present user interface. With a single tap of the stylus
[0073] Using the up/down ones of buttons
[0074] As indicated above, in one embodiment when the user selects the logo icon
[0075] When the folder menu
[0076] From the folder menu
[0077] Anytime the user selects the logo icon
[0078] The user can open any folder or launch any recently used application by appropriate selection operations from the folder menu
[0079] The up/down (or rocker navigation) ones of buttons
[0080] The user can return the navigation flow to the folder menu level by pressing the center one of buttons
[0081] For ease-of-use, like applications can be accessed on the PDA telephone
TABLE 3 Folder Menu Groups and Application Contents Menu Group Applications All All applications on the PDA telephone 100. Main A PIM (e.g., Contacts, Calendar/Data Book/Schedule, Tasks/ To Do/Note Pad), an account manager (e.g., Microsoft Money ™), various Time applications (e.g., a world time clock), Alarm/Stopwatch, Calculator, Drawing Pad/Image Editor, and Smart Card Reader/Writer Application, etc. Com- Telephone dialer, SMS, E-mail, Instant Messenger, Facsimile muni- send/receive application, Internet Browser, Wireless Access cations Protocol Browser, Bluetooth Communication and File Transfer Agent, etc. Appli- Personal productivity applications such as a Text Editor, Spread- cations sheet, Database, Presentation Application, E-Book Reader, Image Viewer/Editor, Calculator, Audio Manager, etc. Multi- Various Media Player(s) (e.g., MIP3, AVI, ASF, MPEG, media Quicktime ™, etc.), Audio Manager, Macromedia ™ FLASH Player, and RealPlayer ™, etc. Games Minesweeper ™, Solitaire, Chess, Tetris ™,