Patent application number | Description | Published |
20080275891 | METHOD TO CREATE A PARTITION-BY TIME/TUPLE-BASED WINDOW IN AN EVENT PROCESSING SERVICE - A method to create a partition by time/tuple based window in an event processing service is provided. When continuous data streams are received, tuples are stored in a data structure with partitions based upon partition keys. Only a specified amount of tuples may be stored in each partition. When a partition exceeds the specified number of tuples, the oldest tuples are removed from the data structure. Tuples stored beyond a specified time period are also removed from the data structure. Two data structures may also be used to implement a time/tuple based window. Tuples are stored in both a data structure with a partition by window and a data structure with a range window. Tuples are removed in the partition by window when tuples exceed the amount in the partition. Tuples are removed in the range window when tuples exceed a specified time period. The two data structures are synchronized. | 11-06-2008 |
20090037370 | DISTRIBUTED EVENT PROCESSING - Methods and systems for distributed event processing are disclosed. In one embodiment, a system for distributed event processing includes a first event processor and a second event processor. The first event processor receives event data associated with an event stream. The second event processor receives a query associated with the event stream. A request to the first event processor is generated by the second event processor based on the query for a first portion of event data associated with the event stream. The second event processor receives the first portion of event data from the first event processor. The second event processor then generates an event based on the first portion of event data. | 02-05-2009 |
20090100029 | Handling Silent Relations In A Data Stream Management System - A new continuous query to a data stream management system (DSMS) may use a silent relation whose source does not provide any heartbeats. During execution of any given operator, the DSMS processes data from one input (“active input”) using its time stamp as if the same time stamp is received from another input whose relation has fallen silent (“empty input”), if one or more predetermined conditions are met. One predetermined condition is that the empty input's operator has all its own inputs silent. And another predetermined condition is that a maximum time stamp that is currently present among all the inputs of the empty input's operator is equal to a time stamp that was last received from these same inputs. When both conditions are met, previously received data from the empty input is time stamped with the same time stamp as the most recently received data from the active input. | 04-16-2009 |
20090106189 | Dynamically Sharing A Subtree Of Operators In A Data Stream Management System Operating On Existing Queries - A new continuous query to a data stream management system (DSMS) may use several operators that are equivalent to operators currently being used by continuous queries that have been previously registered in the DSMS. To determine equivalence of operators, the DSMS checks at least the function and the data input to the operators. On finding equivalence, the DSMS modifies a global plan being executed, to use at least an existing subtree of operators during execution of the new continuous query, to generate a modified plan. The DSMS is also programmed to cause each relation source operator (which outputs a relation) to propagate a current state of the relation to each operator newly coupled to the relational operator. After propagation of current state to newly-coupled operators, each operator in the modified plan processes any new data and supplies the result to all operators coupled thereto, including newly-coupled operators and existing operators. | 04-23-2009 |
20090106190 | Support For User Defined Functions In A Data Stream Management System - A data stream management system (DSMS) is designed to support a new user-defined function, by creating and using at least two structures as follows. A first structure (“metadata entry”) is created in response to a command for creation of the new function, and maps a single instance of a class to the function's name. A second structure is created with creation of an operator on receipt of each new continuous query that uses the new function. The second structure (“operator specific data structure”) contains a path to the newly-created instance, which path is obtained by looking up the first structure. Additional second structures are created on receipt of additional continuous queries which use the new function, but all second structures contain the same path. All continuous queries use the same instance. Repeated use of a single instance to compile and execute multiple queries eliminates repeated instantiation of the same function. | 04-23-2009 |
20090106198 | Support for sharing computation between aggregations in a data stream management system - A computer is programmed to process a continuous query that is known to perform a new aggregation on one or more stream(s) of data, using one or more other aggregations on the stream(s). The computer creates an operator to execute the continuous query, and schedules the operator for execution in a specific order. In several embodiments, the computer determines the order based on dependency of the new aggregation on other aggregation(s), and on the order of performance of the other aggregation(s). The new aggregation is scheduled for performance after performance of each of the other aggregations. The computer is further programmed to pass results of the other aggregations to the new aggregation, by execution of a predetermined function. Support for use of the other aggregations results within the new aggregation eliminates redundant computation of the other aggregations within the new aggregation. The new aggregation may be user defined or built-in. | 04-23-2009 |
20090106214 | ADDING NEW CONTINUOUS QUERIES TO A DATA STREAM MANAGEMENT SYSTEM OPERATING ON EXISTING QUERIES - A new continuous query to a data stream management system (DSMS) may use a stream or a relation which may or may not be used by continuous queries previously registered in the DSMS. The DSMS is programmed to modify an execution plan to accommodate execution of the new query while continuing to execute the previously registered continuous queries. The modified execution plan may include new operators and/or share existing operators. The DSMS is programmed to cause operators which output a relation to propagate a current state of the relation to each newly-coupled operator that uses the relation. The current state is propagated only to operators that have been newly coupled and have thus not yet received any state information previously. After propagation of current state to newly-coupled operators, results of processing any new data for the relation are supplied to all operators coupled thereto, including newly-coupled operators and existing operators. | 04-23-2009 |
20090106215 | DELETING A CONTINUOUS QUERY FROM A DATA STREAM MANAGEMENT SYSTEM CONTINUING TO OPERATE ON OTHER QUERIES - A first continuous query is deleted from several continuous queries that are being executed, by performing different actions that depend on how resources are shared therebetween. As an example, a second continuous query is deleted if a view is referenced by the first continuous query, and if the view is built on the second continuous query, and if the view is not referenced by any query other than the first continuous query. Such deletion is followed by compiling the second continuous query, followed by updating the view to refer to a newly-compiled version of the second continuous query. As another example, if an operator in the first continuous query is not used by any other query, this operator is de-scheduled from execution, and any exclusively-owned resources are released. Also, data to be read by the operator and data generated by the operator, if present in shared resources, are deleted therefrom. | 04-23-2009 |
20090106218 | Support for user defined aggregations in a data stream management system - A computer is programmed to accept a command to create a new aggregation defined by a user during execution of continuous queries on streams of data. The computer is further programmed to thereafter accept and process new continuous queries using the new aggregation, in a manner similar to built-in aggregations. The user typically writes a set of instructions to perform the new aggregation, and identifies in the command, a location of the set of instructions. In response to such a command, the computer creates metadata identifying the new aggregation. The metadata is used to instantiate one aggregation for each group of data in a current window, grouped by an attribute identified in a new query. | 04-23-2009 |
20090106440 | Support for incrementally processing user defined aggregations in a data stream management system - A computer is programmed to accept a command for creation of a new aggregation defined by a user to process data incrementally, one tuple at a time. One or more incremental function(s) in a set of instructions written by the user to implement the new aggregation maintain(s) locally any information that is to be passed between successive invocations, to support computing the aggregation for a given set of tuples as a whole. The user writes a set of instructions to perform the aggregation incrementally, including a plus function which is repeatedly invoked, only once, for each addition to a window of a message. The user also writes a minus function to be invoked with the message, to return the value of incremental aggregation over the window after removal of the message. In such embodiments, the computer does not maintain copies of messages in the window for use by aggregation function(s). | 04-23-2009 |
20090112802 | Facilitating Flexible Windows In Data Stream Management Systems - Facilitating flexible windows in querying continuous data streams. In one embodiment, a software code is associated with a window, wherein execution of the software code determines the values to be included in the window. On receiving a query indicating the window, the result of the query is generated based on the values in the window determined by execution of the software code. The software code may be identified by an identifier and thus used for different queries on the same or different continuous data streams. | 04-30-2009 |
20090112803 | Specifying A Family Of Logics Defining Windows In Data Stream Management Systems - Simplifying a user's task in specifying a family of logics, with each logic designed to potentially select different subsets of values from the same continuous data stream at the same time instant. In one embodiment, all the logics are together specified by a common user specified software code design to accept a set of parameters, with different logics resulting from execution of the software code with different parametric values. Thus, when closely related family of logics are to be developed, the common software code may be formulated, with the code being designed to accept parameters and providing different logics for different sets of parametric values corresponding to the parameters. | 04-30-2009 |
20100057663 | TECHNIQUES FOR MATCHING A CERTAIN CLASS OF REGULAR EXPRESSION-BASED PATTERNS IN DATA STREAMS - Techniques for detecting patterns in data streams. A pattern to be detected may be specified using a regular expression. Events received in data streams are then processed during runtime to detect occurrences of the pattern specified by the regular expression in the data stream. If the pattern to be matched belongs to a certain predetermined class of patterns, then a pattern matching technique that is customized for that class of patterns is used during the runtime events processing. | 03-04-2010 |
20100057735 | FRAMEWORK FOR SUPPORTING REGULAR EXPRESSION-BASED PATTERN MATCHING IN DATA STREAMS - Techniques for detecting patterns in one or more data or event streams. A pattern to be detected may be specified using a regular expression. Events received in a data stream are processed during runtime to detect occurrences of the specified pattern in the data stream. In one embodiment, a pattern type or class is determined for the specified pattern and pattern matching is performed using a technique selected based upon the type or class determined for the specified pattern. | 03-04-2010 |
20100057736 | TECHNIQUES FOR PERFORMING REGULAR EXPRESSION-BASED PATTERN MATCHING IN DATA STREAMS - Techniques for detecting patterns in one or more data streams. A pattern to be detected may be specified using a regular expression. Events received in a data stream are processed during runtime to detect occurrences of the specified pattern in the data stream. | 03-04-2010 |
20100223305 | INFRASTRUCTURE FOR SPILLING PAGES TO A PERSISTENT STORE - Techniques for managing memory usage in a processing system are provided. This may be achieved by receiving a data stream including multiple tuples and determining a query plan that was generated for a continuous query applied to the multiple tuples in the data stream. The query plan may include one or more operators. Before scheduling an operator in the query plan, it is determined when an eviction is to be performed based a level of free memory of the processing system. An eviction candidate is determined and a page associated with the eviction candidate is evicted from the memory to a persistent storage. | 09-02-2010 |
20100223437 | METHOD AND SYSTEM FOR SPILLING FROM A QUEUE TO A PERSISTENT STORE - Techniques for managing memory usage of a processing system by spilling data from a memory to a persistent store based upon an evict policy are provided. A triggering event is detected. In response to the triggering event and based on the evict policy, it is determined whether data from the memory of the processing system is to be spilled to the persistent storage. The determination is made by comparing a level of free memory of the processing system with a threshold specified by the evict policy. The data is evicted from the memory. | 09-02-2010 |
20100223606 | FRAMEWORK FOR DYNAMICALLY GENERATING TUPLE AND PAGE CLASSES - Techniques for reducing the memory used for processing events received in a data stream are provided. This may be achieved by reducing the memory required for storing tuples. A method for processing a data stream includes receiving a tuple and determining a tuple specification that defines a layout of the tuple. The layout identifies one or more data types that are included in the tuple. A tuple class corresponding to the tuple specification may be determined. A tuple object based on the tuple class is instantiated, and during runtime of the processing system. The tuple object is stored in a memory. | 09-02-2010 |
20110022618 | STANDARDIZED DATABASE CONNECTIVITY SUPPORT FOR AN EVENT PROCESSING SERVER IN AN EMBEDDED CONTEXT - Techniques for facilitating communication between one or more client applications and an event processing server. In one set of embodiments, a framework is provided that enables a client application to communicate with an event processing server via a standard database connectivity API, such as Java Database Connectivity (JDBC). | 01-27-2011 |
20110023055 | STANDARDIZED DATABASE CONNECTIVITY SUPPORT FOR AN EVENT PROCESSING SERVER - Techniques for facilitating communication between one or more client applications and an event processing server. In one set of embodiments, a framework is provided that enables a client application to communicate with an event processing server via a standard database connectivity API, such as Java Database Connectivity (JDBC). | 01-27-2011 |
20110029484 | LOGGING FRAMEWORK FOR A DATA STREAM PROCESSING SERVER - Techniques for logging data pertaining to the operation of a data stream processing server. In one set of embodiments, logging configuration information can be received specifying a functional area of a data stream processing server to be logged. Based on the logging configuration information, logging can be dynamically enabled for objects associated with the functional area that are instantiated by the data stream processing server, and logging can be dynamically disabled for objects associated with the functional area that are discarded (or no longer used) by the data stream processing server. In another set of embodiments, a tool can be provided for visualizing the data logged by the data stream processing server. | 02-03-2011 |
20110029485 | LOG VISUALIZATION TOOL FOR A DATA STREAM PROCESSING SERVER - Techniques for logging data pertaining to the operation of a data stream processing server. In one set of embodiments, logging configuration information can be received specifying a functional area of a data stream processing server to be logged. Based on the logging configuration information, logging can be dynamically enabled for objects associated with the functional area that are instantiated by the data stream processing server, and logging can be dynamically disabled for objects associated with the functional area that are discarded (or no longer used) by the data stream processing server. In another set of embodiments, a tool can be provided for visualizing the data logged by the data stream processing server. | 02-03-2011 |
20110173182 | COMPILE TIME OPCODES FOR EFFICIENT RUNTIME INTERPRETATION OF VARIABLES FOR DATABASE QUERIES AND DML STATEMENTS - A method system and program for performing a query in which context and information including opcodes accompanies an execution plan for the query. The opcodes determine operations to retrieve a set of bind variables. The execution plan and context information shared by a plurality of application programs. | 07-14-2011 |
20110246445 | METHODS AND SYSTEMS FOR QUERYING EVENT STREAMS USING MULTIPLE EVENT PROCESSORS - In one embodiment, a system for distributed event processing includes a first event processor and a second event processor. The first event processor receives event data associated with an event stream. The second event processor receives a query associated with the event stream. A request to the first event processor is generated by the second event processor based on the query for a first portion of event data associated with the event stream. The second event processor receives the first portion of event data from the first event processor. The second event processor then generates an event based on the first portion of event data. | 10-06-2011 |
20110270879 | SUPPORT FOR USER DEFINED AGGREGATIONS IN A DATA STREAM MANAGEMENT SYSTEM - A computer is programmed to accept a command to create a new aggregation defined by a user during execution of continuous queries on streams of data. The computer is further programmed to thereafter accept and process new continuous queries using the new aggregation, in a manner similar to built-in aggregations. The user typically writes a set of instructions to perform the new aggregation, and identifies in the command, a location of the set of instructions. In response to such a command, the computer creates metadata identifying the new aggregation. The metadata is used to instantiate one aggregation for each group of data in a current window, grouped by an attribute identified in a new query. | 11-03-2011 |
20120041934 | SUPPORT FOR USER DEFINED FUNCTIONS IN A DATA STREAM MANAGEMENT SYSTEM - A data stream management system (DSMS) is designed to support a new user-defined function, by creating and using at least two structures as follows. A first structure (“metadata entry”) is created in response to a command for creation of the new function, and maps a single instance of a class to the function's name. A second structure is created with creation of an operator on receipt of each new continuous query that uses the new function. The second structure (“operator specific data structure”) contains a path to the newly-created instance, which path is obtained by looking up the first structure. Additional second structures are created on receipt of additional continuous queries which use the new function, but all second structures contain the same path. All continuous queries use the same instance. Repeated use of a single instance to compile and execute multiple queries eliminates repeated instantiation of the same function. | 02-16-2012 |