Patent application number | Description | Published |
20080306920 | BYTE-CODE REPRESENTATIONS OF ACTUAL DATA TO REDUCE NETWORK TRAFFIC IN DATABASE TRANSACTIONS - Methods, articles of manufacture, and systems for reducing network traffic for database transactions between a requesting client and a database server are provided. For some embodiments, the client may access statistical information regarding the data, such as that commonly contained in indexes (e.g., encoded vector indexes), conventionally generated for use by a query optimizer. For some embodiments, based on the statistical information regarding the data, the client may request that the server send small byte-code representations of data from the database, rather than the actual data. The small byte-code representations may be significantly smaller than the actual represented data, thus reducing network traffic between the client and server. The client may then retrieve the actual represented data from a corresponding data representation record. | 12-11-2008 |
20080320450 | RECOVERABLE RETURN CODE TRACKING AND NOTIFICATION FOR AUTONOMIC SYSTEMS - A system, method and article of manufacture return code management in autonomic systems and more particularly to managing execution of operations in data processing systems on the basis of return code tracking. One embodiment provides a method for managing execution of an operation in a data processing system. The method comprises tracking return codes received from previous executions of the operation in the data processing system, determining an execution behavior of the operation from the tracked return codes, and managing a subsequent execution of the operation on the basis of the determined execution behavior. | 12-25-2008 |
20090043728 | Query Optimization in a Parallel Computer System to Reduce Network Traffic - An apparatus and method for a database query optimizer to optimize a query that uses multiple networks. The query optimizer optimizes a query to reduce network traffic on a network or node that is overloaded or above an established parameter in a node/network attribute table. The query optimization to reduce network traffic may result in a sub-optimal query in other respects such as execution time. The result is a query optimizer that rewrites or optimizes a query to execute on multiple nodes or networks to reduce traffic on a network or node according to the loading characteristics and assigned attributes of a node or network. | 02-12-2009 |
20090043745 | Query Execution and Optimization with Autonomic Error Recovery from Network Failures in a Parallel Computer System with Multiple Networks - An apparatus and method for a database query execution monitor determines if an network error or low performance condition exists and then where possible modifies the query. The query execution monitor then determines an alternate query execution plan to continue execution of the query. The query optimizer can re-optimize the query to use a different network or node. Thus, the query execution monitor allows autonomic error recovery for network failures using an alternate query execution. The alternate query execution could also be determined at the initial optimization time and then this alternate plan used to execute a query in the case of a particular network failure. | 02-12-2009 |
20090043750 | Query Optimization in a Parallel Computer System with Multiple Networks - An apparatus and method for a database query optimizer to optimize a query that uses multiple networks. The database query optimizer optimizes a query that uses multiple networks to satisfy the query by splitting the query execution to use multiple networks. Thus, the query optimizer rewrites or optimizes a query to execute on multiple nodes or networks to more efficiently execute the query and reduce network traffic on a network. The query optimizer uses plan cache statistics to determine whether to use multiple networks to optimize the query. | 02-12-2009 |
20090043873 | Methods and Apparatus for Restoring a Node State - In one aspect of the invention, a method is provided. The method may include: (1) storing a snapshot of a system state of a node; (2) executing a job on the node; and (3) restoring the node to the system state using the stored snapshot of the system state. | 02-12-2009 |
20090043910 | Query Execution and Optimization Utilizing a Combining Network in a Parallel Computer System - An apparatus and method for a database query optimizer utilizes a combining network to optimize a portion of a query in a parallel computer system with multiple nodes. The efficiency of the parallel computer system is increased by offloading collective operations on node data to the global combining network. The global combining network performs collective operations such as minimum, maximum, sum, and logical functions such as OR and XOR. | 02-12-2009 |
20090089743 | Methods and Apparatus for IDE Integration of Database Changes - In one aspect, a method is provided. The method includes (1) generating code in an integrated development environment (IDE), the code including instructions for a database; and (2) receiving feedback in the IDE, the feedback relating to the instructions for the database. | 04-02-2009 |
20090100195 | Methods and Apparatus for Autonomic Compression Level Selection for Backup Environments - In one aspect, a method is provided. The method includes: (1) gathering statistics during compression of a dataset into a compressed dataset and during transfer of the compressed dataset over a network connection; and (2) optimizing compression settings based on the gathered statistics. | 04-16-2009 |
20090125611 | SHARING LOADED JAVA CLASSES AMONG A PLURALITY OF NODES - Methods, apparatus, and products are disclosed for sharing loaded Java classes among a plurality of nodes connected together for data communications using a data communication network, the plurality of nodes including an execution node and other nodes, that include: executing, by the execution node, a Java application, including identifying a Java class utilized for the Java application; determining, by the execution node, whether the Java class is already loaded on at least one of the other nodes; retrieving, by the execution node, the loaded Java class from the other nodes if the Java class is already loaded on at least one of the other nodes; and executing, by the execution node, the Java application using the loaded Java class retrieved from the other nodes. | 05-14-2009 |
20090125883 | NODE SELECTION FOR EXECUTING A JAVA APPLICATION AMONG A PLURALITY OF NODES - Methods, apparatus, and products are disclosed for node selection for executing a Java application among a plurality of nodes connected together for data communications using a data communication network, the plurality of nodes also connected to a service node, that include: tracking, by the service node, loaded Java classes currently loaded on each of the plurality of nodes; receiving, in the service node, an instruction to execute a Java application using the plurality of nodes; identifying, by the service node, Java classes utilized in executing the Java application; selecting, by the service node, one of the plurality of nodes for executing the Java application in dependence upon the loaded Java classes and the Java classes utilized in executing the Java application; and configuring, by the service node, the Java application for execution on the selected node. | 05-14-2009 |
20090182705 | Methods and Apparatus for Converting Union Commands to Union All Commands - In one aspect, a method is provided. The method includes: (1) identifying in a query of a database a union command involving data in the database that is mutually exclusive; and (2) changing the union command to a union all command. | 07-16-2009 |
20090240930 | Executing An Application On A Parallel Computer - Methods, systems, and products are disclosed for executing an application on a parallel computer including a plurality of nodes connected together through a data communications network. Each node has a plurality of processors capable of operating independently for serial processing and capable of operating symmetrically for parallel processing. The application has parallel segments for parallel processing and serial segments for serial processing. Embodiments of the invention include: booting up a first subset of the plurality of nodes in a serial processing mode; booting up a second subset of the plurality of nodes in a parallel processing mode; and executing the application on the plurality of nodes, including: migrating the application to the nodes booted up in the parallel processing mode upon encountering the parallel segments during execution, and migrating the application to the nodes booted up in the serial processing mode upon encountering the serial segments during execution. | 09-24-2009 |
20090265305 | Partial Indexes for Multi-Node Database - Embodiments of the invention enable a database spread over multiple nodes to allow each node to have different indexes over the data in tables, depending on how each node would benefit (or not benefit) from having the index(es). When a database table is spread across the nodes of a multi-node or distributed system, each node may maintain only the portion of the index relevant to that node, if doing so would improve the performance of query processing operations on that node. Further, the database may periodically redistributed across the compute nodes based on index performance. Doing so allows the database system to intelligently trade off between consuming space for the index on a node and the usefulness of having an index on that node. | 10-22-2009 |
20090265306 | Index Maintenance in a Multi-Node Database - Embodiments of the invention enable a database spread over multiple nodes to allow each node to have different indexes over the data in tables, depending on how each node would benefit (or not benefit) from having the index(es). When a database table is spread across the nodes of a multi-node or distributed system, each node may maintain only the portion of the index relevant to that node, if doing so would improve the performance of query processing operations on that node. Further, the database may periodically redistributed across the compute nodes based on index performance. Doing so allows the database system to intelligently trade off between consuming space for the index on a node and the usefulness of having an index on that node. | 10-22-2009 |
20090271588 | Migrating Program Objects in a Multi-Node Computer System - Embodiments of the invention enable application programs running across multiple compute nodes of a highly-parallel system to selectively migrate objects from one node to another. For example, when an object becomes too large, a node containing the object may migrate the object to another node, thereby freeing memory space. Whether a large object is migrated can be dependent on how frequently the object is used by the application. Because the memory used by such an object is freed for other uses by the application, overall application performance may be improved. On large parallel systems with thousands of compute nodes, even relatively small improvements in application performance an individual compute node may be magnified many times, resulting in dramatic improvements in overall application performance. | 10-29-2009 |
20090271595 | Configuring An Application For Execution On A Parallel Computer - Methods, systems, and products are disclosed for configuring an application for execution on a parallel computer that include: booting up a first subset of a plurality of nodes in a serial processing mode; booting up a second subset of the plurality of nodes in a parallel processing mode; profiling, prior to application deployment on the parallel computer, the application to identify the serial segments and the parallel segments of the application; and deploying the application for execution on the parallel computer in dependence upon the profile of the application and proximity within the data communications network of the nodes in the first subset relative to the nodes in the second subset. | 10-29-2009 |
20090271596 | Executing An Application On A Parallel Computer - Methods, systems, and products are disclosed for executing an application on a parallel computer having a plurality of nodes. Executing an application on a parallel computer includes: booting up a first subset of a plurality of nodes in a serial processing mode; booting up a second subset of the plurality of nodes in a parallel processing mode; profiling, prior to application execution, an application to identify serial segments of the application, parallel segments of the application, and application data utilized by each of the serial segments and the parallel segments; and executing the application on the plurality of nodes, including migrating, in dependence upon the profile for the application upon encountering the parallel segments during execution, only specific portions of the application and the application data from the nodes booted up in the serial processing mode to the nodes booted up in the parallel processing mode. | 10-29-2009 |
20090271775 | Optimizing Just-In-Time Compiling For A Java Application Executing On A Compute Node - Methods, systems, and products are disclosed for optimizing just-in-time (‘JIT’) compiling for a Java application executing on a compute node, the compute node having installed upon it a Java Virtual Machine (‘JVM’) capable of supporting the Java application, that include: identifying, by an application manager, a particular portion of the Java application; assigning, by the application manager, a JIT level to the particular portion of the Java application; and jitting, by the JVM installed on the compute node, the particular portion of the Java application in dependence upon the JIT level assigned to that particular portion of the Java application. | 10-29-2009 |
20090271784 | Executing A Distributed Java Application On A Plurality Of Compute Nodes - Methods, systems, and products are disclosed for executing a distributed Java application on a plurality of compute nodes. The Java application includes a plurality of jobs distributed among the plurality of compute nodes. The plurality of compute nodes are connected together for data communications through a data communication network. Each of the plurality of compute nodes has installed upon it a Java Virtual Machine (‘JVM’) capable of supporting at least one job of the Java application. Executing a distributed Java application on a plurality of compute nodes includes: tracking, by an application manager, JVM environment variables for the JVMs installed on the plurality of compute nodes; and configuring, by the application manager, the plurality of jobs for execution on the plurality of compute nodes in dependence upon the JVM environment variables for the JVMs installed on the plurality of compute nodes. | 10-29-2009 |
20090271799 | Executing A Distributed Java Application On A Plurality Of Compute Nodes - Methods, systems, and products are disclosed for executing a distributed Java application on a plurality of compute nodes. The Java application includes a plurality of jobs distributed among the plurality of compute nodes. The plurality of compute nodes are connected together for data communications through a data communication network. Each of the plurality of compute nodes has installed upon it a Java Virtual Machine (‘JVM’) capable of supporting at least one job of the Java application. Executing a distributed Java application on a plurality of compute nodes includes: tracking, by an application manager, a just-in-time (‘JIT’) compilation history for the JVMs installed on the plurality of compute nodes; and configuring, by the application manager, the plurality of jobs for execution on the plurality of compute nodes in dependence upon the JIT compilation history for the JVMs installed on the plurality of compute nodes. | 10-29-2009 |
20090271807 | Selectively Generating Program Objects on Remote Node of a Multi-Node Computer System - Embodiments of the invention enable application programs running across multiple compute nodes of a highly-parallel system to selectively migrate objects from one node to another. For example, when an object becomes too large, a node containing the object may migrate the object to another node, thereby freeing memory space. Whether a large object is migrated can be dependent on how frequently the object is used by the application. Because the memory used by such an object is freed for other uses by the application, overall application performance may be improved. On large parallel systems with thousands of compute nodes, even relatively small improvements in application performance an individual compute node may be magnified many times, resulting in dramatic improvements in overall application performance. | 10-29-2009 |
20090299667 | Qualifying Data Produced By An Application Carried Out Using A Plurality Of Pluggable Processing Components - Methods, apparatus, and products are disclosed for qualifying data produced by an application carried out using a plurality of pluggable processing components. Qualifying data produced by the application includes: receiving, by an application manager, quality metrics for one of the pluggable processing components; determining, by the application manager, a component quality rating for the pluggable processing component in dependence upon the quality metrics; and assigning, by the application manager, a data quality rating to application data for the application in dependence upon the component quality rating for the pluggable processing component. | 12-03-2009 |
20090300154 | Managing performance of a job performed in a distributed computing system - Methods, systems, and products are disclosed for managing performance of a job performed in a distributed computing system, the distributed computing system comprising a plurality of compute nodes operatively coupled through a data communications network, the job carried out by a plurality of distributed pluggable processing components executing on the plurality of compute nodes, that include: identifying a current configuration of the pluggable processing components carrying out the job, the current configuration specifying a current distribution of the pluggable processing components among the compute nodes; identifying a network topology of the plurality of compute nodes in the data communications network; receiving a plurality of performance indicators produced during execution of the job; and redistributing, to a different compute node, at least one of the pluggable processing components in dependence upon the current configuration, the network topology, and the performance indicators. | 12-03-2009 |
20090300404 | Managing Execution Stability Of An Application Carried Out Using A Plurality Of Pluggable Processing Components - Methods, apparatus, and products are disclosed for managing execution stability of an application carried out using a plurality of pluggable processing components. Managing execution stability of an application includes: receiving, by an application manager, component stability metrics for a particular pluggable processing component; determining, by the application manager, that the particular pluggable processing component is unstable in dependence upon the component stability metrics for the particular pluggable processing component; and notifying, by the application manager, a system administrator that the particular pluggable processing component is unstable. | 12-03-2009 |
20090300624 | Tracking data processing in an application carried out on a distributed computing system - Methods, systems, and products are disclosed for tracking data processing in an application carried out on a distributed computing system, the distributed computing system including a plurality of computing nodes connected through a data communications network, the application carried out by a plurality of pluggable processing components installed on the plurality of computing nodes, the pluggable processing components including a pluggable processing provider component and a pluggable processing consumer component, that include: identifying, by the provider component, data satisfying predetermined processing criteria, the criteria specifying that the data is relevant to processing provided by the consumer component; passing, by the provider component, the data to the next pluggable processing component in the application for processing, including maintaining access to the data; receiving, by the consumer component, the data during execution of the application; and sending, by the consumer component, a receipt indicating that the consumer component received the data. | 12-03-2009 |
20090300625 | Managing The Performance Of An Application Carried Out Using A Plurality Of Pluggable Processing Components - Methods, apparatus, and products are disclosed for managing the performance of an application carried out using a plurality of pluggable processing components, the pluggable processing components executed on a plurality of compute nodes, that include: identifying a current configuration of the pluggable processing components for carrying out the application; receiving a plurality of performance indicators produced during execution of the pluggable processing components; and altering the current configuration of the pluggable processing components in dependence upon the performance indicators and one or more additional pluggable processing components. | 12-03-2009 |
20090313636 | Executing An Application On A Parallel Computer - Methods, apparatus, and products are disclosed for executing an application on a parallel computer that include: executing, by a current compute node, a current task of the application, including producing results; determining, by the current compute node in dependence upon current network characteristics and application characteristics, whether to transfer the results to a next compute node for further processing by a next task on the next compute node or to execute the next task for further processing of the results on the current compute node; transferring, by the current compute node, the results to the next compute node for further processing by the next task on the next compute node if the determination specifies transferring the results to the next node; and executing, by the current compute node, the next task for further processing of the results if the determination specifies executing the next task on the current compute node. | 12-17-2009 |
20090319621 | Message Flow Control in a Multi-Node Computer System - Embodiments of the invention provide for controlling message flow across a parallel computer system having multiple compute nodes by selectively grouping compute nodes of such a system into node pools and assigning message flow control policies to nodes in the node pools. The message flow control policies specify logging and/or tracing activities to be performed by instances of applications running on nodes assigned to the node pools. As the application is executed, logging and/or tracing messages are generated on the compute nodes according to message flow control policies assigned to the nodes. Optionally, the message flow is analyzed, the message flow control policies are adjusted, and duplicate messages are eliminated. | 12-24-2009 |
20090319662 | Process Migration Based on Exception Handling in a Multi-Node Environment - A process on a highly distributed parallel computing system is disclosed. When a first compute node in a first pool is ready to hand-off a task to second pool for further processing, the first compute node may first determine whether a node is available in the second pool. If no node is available from the second pool, then the first compute node may begin performing a primary task assigned to the second pool of nodes, up to the point where a service available exclusively to the nodes of the second pool is required. In the interim, however, one of the nodes of the second pool may become available. Alternatively, an application program running on a compute node may be configured with an exception handling routine that catches exceptions and migrates the application to a compute node where a necessary service is available, as such exceptions occur. | 12-24-2009 |
20090320023 | Process Migration Based on Service Availability in a Multi-Node Environment - A process on a highly distributed parallel computing system is disclosed. When a first compute node in a first pool is ready to hand-off a task to second pool for further processing, the first compute node may first determine whether a node is available in the second pool. If no node is available from the second pool, then the first compute node may begin performing a primary task assigned to the second pool of nodes, up to the point where a service available exclusively to the nodes of the second pool is required. In the interim, however, one of the nodes of the second pool may become available. Alternatively, an application program running on a compute node may be configured with an exception handling routine that catches exceptions and migrates the application to a compute node where a necessary service is available, as such exceptions occur. | 12-24-2009 |
20100085870 | GLOBAL DETECTION OF RESOURCE LEAKS IN A MULTI-NODE COMPUTER SYSTEM - A process is disclosed for identifying and recovering from resource leaks on compute nodes of a parallel computing system. A resource monitor stores information about system resources available on a compute node in a clean state. After the compute node runs a job, the resource monitor compares the current resource availability to the clean state. If a resource leak is found, the resource monitor contacts a global resource manger to remove the resource leak. | 04-08-2010 |
20100085871 | RESOURCE LEAK RECOVERY IN A MULTI-NODE COMPUTER SYSTEM - A process is disclosed for identifying and recovering from resource leaks on compute nodes of a parallel computing system. A resource monitor stores information about system resources available on a compute node in a clean state. After the compute node runs a job, the resource monitor compares the current resource availability to the clean state. If a resource leak is found, the resource monitor contacts a global resource manger to remove the resource leak. | 04-08-2010 |
20100180024 | REDUCING OCCURRENCES OF TWO-PHASE COMMITS IN A MULTI-NODE COMPUTING SYSTEM - Techniques are disclosed for reducing the number of two-phase commits required to perform atomic transactions in a multi-node system. By monitoring atomic operations that require two-phase commit protocols, a system may identify when it would be beneficial to consolidate data onto single (or at least fewer) compute node(s). In doing so, a less compute intensive commit protocol, such as a one-phase commit protocol may be used. Moreover, in situations where data consolidation is not available, the data may be migrated to compute nodes having a closer proximity. Thus, in such cases, network response times for two-phase commits required for an atomic operation may be reduced, thereby increasing the performance of a multi-node system. | 07-15-2010 |
20100205578 | PROGRAM DEVELOPMENT TOOL CONFIGURED TO COMPILE SOURCE CODE ACCORDING TO ENERGY CONSUMPTION REQUIREMENTS - Embodiments of the invention provide techniques for presenting energy consumption information in an IDE tool. In one embodiment, the IDE tool may be configured to determine energy requirements associated with specific elements of the source code, and to present graphical indications of energy requirements along with those code elements. Such energy requirements may be determined by matching code elements to a stored data structure describing energy requirements for executing various code elements. The stored data may be based on predefined standards, and/or may be based on historical measurements of energy consumption during prior instances of executing code elements on a target computer system. Additionally, developers may specify priority for compiling portions of source code, according to desired energy requirements. | 08-12-2010 |
20100205591 | PRESENTING ENERGY CONSUMPTION INFORMATION IN AN INTEGRATED DEVELOPMENT ENVIRONMENT TOOL - Embodiments of the invention provide techniques for presenting energy consumption information in an IDE tool. In one embodiment, the IDE tool may be configured to determine energy requirements associated with specific elements of the source code, and to present graphical indications of energy requirements along with those code elements. Such energy requirements may be determined by matching code elements to a stored data structure describing energy requirements for executing various code elements. The stored data may be based on predefined standards, and/or may be based on historical measurements of energy consumption during prior instances of executing code elements on a target computer system. Additionally, developers may specify priority for compiling portions of source code, according to desired energy requirements. | 08-12-2010 |
20100205619 | ENVIRONMENT MODIFICATION IN A HYBRID NODE COMPUTING ENVIRONMENT - Embodiments of the invention provide techniques that improve resource management on a massively parallel computing system having a plurality of hybrid compute nodes. For example, a job scheduler may be provided which determines a library to link to an application based on system and user requirements. In one embodiment, the libraries may provide optimizations for job execution time, and also provide optimizations directed towards a specific processor architecture. Once the library is determined, the job scheduler may configure the environment of the application so that the application links with the optimized library during run-time. Doing so may improve overall system performance of the massively parallel computing system. | 08-12-2010 |
20100242026 | ENHANCED THREAD STEPPING - Embodiments of the invention provide debugging techniques for multithreaded programs. For example, a debugger may reduce the time required to release inactive threads that hold resources needed by an active thread. For example, if a timer expires before a line of code finishes executing, the debugger assumes that the active thread requires a resource locked by an inactive thread and releases other threads. During subsequent execution of the line of code, the debugger reduces the timer length of the timer. In doing so, inactive threads are released sooner, thereby reducing the wait time a user experiences when debugging lines of code. | 09-23-2010 |
20110302659 | DATA SECURITY IN A MULTI-NODAL ENVIRONMENT - A data security manager in a multi-nodal environment enforces processing constraints stored as security relationships that control how different pieces of a multi-nodal application (called execution units) are allowed to execute to insure data security. The security manager preferably checks the security relationships for security violations when new execution units start execution, when data moves to or from an execution unit, and when an execution unit requests external services. Where the security manager determines there is a security violation based on the security relationships, the security manager may move, delay or kill an execution unit to maintain data security. | 12-08-2011 |
20110321056 | DYNAMIC RUN TIME ALLOCATION OF DISTRIBUTED JOBS - A job optimizer dynamically changes the allocation of processing units on a multi-nodal computer system. A distributed application is organized as a set of connected processing units. The arrangement of the processing units is dynamically changed at run time to optimize system resources and interprocess communication. A collector collects metrics of the system, nodes, application, jobs and processing units that will be used to determine how to best allocate the jobs on the system. A job optimizer analyzes the collected metrics to dynamically arrange the processing units within the jobs. The job optimizer may determine to combine multiple processing units into a job on a single node when there is an overutilization of interprocess communication between processing units. Alternatively, the job optimizer may determine to split a job's processing units into multiple jobs on different nodes where the processing units are over utilizing the resources on the node. | 12-29-2011 |
20120017218 | DYNAMIC RUN TIME ALLOCATION OF DISTRIBUTED JOBS WITH APPLICATION SPECIFIC METRICS - A job optimizer dynamically changes the allocation of processing units on a multi-nodal computer system. A distributed application is organized as a set of connected processing units. The arrangement of the processing units is dynamically changed at run time to optimize system resources and interprocess communication. A collector collects application specific metrics determined by application plug-ins. A job optimizer analyzes the collected metrics and determines how to dynamically arrange the processing units within the jobs. The job optimizer may determine to combine multiple processing units into a job on a single node when there is an overutilization of an interprocess communication between processing units. Alternatively, the job optimizer may determine to split a job's processing units into multiple jobs on different nodes where one or more of the processing units are over utilizing the resources on the node. | 01-19-2012 |
20120047101 | DETECTING DISALLOWED COMBINATIONS OF DATA WITHIN A PROCESSING ELEMENT - Techniques are described for detecting disallowed combinations of data within a processing element. Embodiments of the invention may generally receive data to be processed using the processing element and determine whether the received data and a current working state violate one or more rules describing disallowed combinations of data. If a disallowed combination is detected, embodiments of the invention may handle the processing of the received data in an alternate way that prevents disallowed combinations of data within the processing element. | 02-23-2012 |
20120047505 | PREDICTIVE REMOVAL OF RUNTIME DATA USING ATTRIBUTE CHARACTERIZING - Techniques are described for selectively removing runtime data from a stream-based application in a manner that reduces the impact of any delay caused by the processing of the data in the stream-based application. In addition to removing the data from a primary processing path of the stream-based application, the data may be processed in an alternate manner, either using alternate processing resources, or by delaying the processing of the data. | 02-23-2012 |
20120066184 | SPECULATIVE EXECUTION IN A REAL-TIME DATA ENVIRONMENT - Techniques are described for speculatively executing operations on data in a data stream in parallel in a manner that increases the efficiency of the stream-based application. In addition to executing operations in parallel, embodiments of the invention may determine whether certain results produced by the parallel operations are valid results and discard any results determined to be invalid. | 03-15-2012 |
20120109908 | ACTIVE MEMORY EXPANSION AND RDBMS META DATA AND TOOLING - Techniques are described for estimating and managing memory compression for query processing. Embodiments of the invention may generally include receiving a query to be executed, ascertaining indicatory data about the retrieved data, and selectively compressing a portion of the data in memory according to the indicatory data. In one embodiment, the amount of compression performed during each query execution is recorded and outputted to assist in adjusting the selective compression process. | 05-03-2012 |
20120110042 | DATABASE INSERTIONS IN A STREAM DATABASE ENVIRONMENT - Methods and systems for storing stream data in a stream application are disclosed. The stream application is composed from a plurality of processing elements executing on one or more compute nodes. The plurality of processing elements is communicatively coupled to a management system. The management system is configured to monitor and control operations of the plurality of processing elements. A trigger rule, which includes a trigger condition, is loaded on a selected processing element in the plurality of process elements. The stream data is then monitored at the first processing element to determine if the trigger condition relative to the stream data has occurred. The stream data is stored in a database when the trigger condition occurs. The stream data is stored in the database according to the trigger rule. | 05-03-2012 |
20120110182 | DYNAMIC PROCESSING UNIT RELOCATION IN A MULTI-NODAL ENVIRONMENT BASED ON INCOMING PHYSICAL DATA - A relocation mechanism in a multi-nodal computer environment dynamically routes processing units in a distributed computer system based on incoming physical data into the processing unit. The relocation mechanism makes an initial location decision to place a processing unit onto a node in the distributed computer system. The relocation mechanism monitors physical data flowing into a processing unit or node and dynamically relocates the processing unit to another type of node within the ‘cloud’ of nodes based on the type of physical data or pattern of data flowing into the processing unit. The relocation mechanism may use one or more rules with criteria for different data types observed in the data flow to optimize when to relocate the processing units. | 05-03-2012 |
20120124002 | REDUCING STORAGE COSTS ASSOCIATED WITH BACKING UP A DATABASE - Techniques are disclosed for backing up and/or restoring data. In one embodiment, a request is received to back up at least a first unit of data stored in a database. Upon determining that the first unit of data is at least partially derived from a second unit of data stored in the database, a backup may be generated that includes less than all of the first unit of data. Subsequently, the first unit of data may be fully restored from the backup. | 05-17-2012 |
20120124016 | ACTIVE MEMORY EXPANSION IN A DATABASE ENVIRONMENT TO QUERY NEEDED/UNEEDED RESULTS - Techniques are described for estimating and managing memory compression for request processing. Embodiments of the invention may generally include receiving a request for data, determining if the requested data contains any compressed data, and sending the requesting entity only the uncompressed data. A separate embodiment generally includes receiving a request for data, determining if the requested data contains any compressed data, gathering uncompression criteria about the requested data, and using the uncompression criteria to selectively determine what portion of the compressed data to uncompress. | 05-17-2012 |
20120131108 | INTELLIGENT OFFLOAD OF WORK TO HANDLE PEAK ACTIVITY IN AN ENTERPRISE EMAIL SYSTEM - A method and system for sending a message are disclosed. The method involves receiving a message at a mail server and determining that a load of the mail server exceeds a threshold. If it is determined that the load of the mail server exceeds a threshold, then the method involves determining whether the message needs to be sent immediately. If it is determined that the message does not need to be sent immediately, then the method involves delaying sending the message. | 05-24-2012 |
20120136850 | MEMORY USAGE QUERY GOVERNOR - Techniques for managing query execution based on memory usage using memory compression. Embodiments may receive a query from a requesting entity and estimate an amount of memory required to execute the query. Upon determining that executing the query would consume an impermissible amount of memory, embodiments may optimize the query by adjusting an amount of memory compression used in executing the query. The optimized query may then be executed to produce a set of query results, which may then be returned to the requesting entity. | 05-31-2012 |
20120137178 | STREAM BASED DEBUGGING TECHNIQUES - Techniques are described for debugging a processing element (or elements) in a stream based database application in a manner that reduces the impact of debugging the processing element (or elements) on the overall running environment by selectively fusing (or un-fusing) processing elements running on a group of compute nodes. In addition to fusing and un-fusing processing elements or otherwise modifying a state of the stream application, a debugging application and stream manager may modify data flows within the application stream in a variety of ways to minimize any disruption resulting from a debugging session. | 05-31-2012 |
20120150806 | NORMALIZING DATA ON DATABASE RESTORE - Techniques for normalizing a database as part of a database restore. Embodiments may receive a database restore request indicating a previous state to restore a database to. Responsive to the request, embodiments may restore the database to the previous state using backup data associated with the previous state, and normalize the restored database using historical database usage data based on one or more previous database operations. | 06-14-2012 |
20120150895 | MAXIMUM ALLOWABLE RUNTIME QUERY GOVERNOR - Techniques for managing query execution using memory compression based on maximum allowable runtimes. Embodiments may receive a query from a requesting entity and calculate an estimated execution time for executing the query. Embodiments may further determine a maximum allowable runtime for the query. An amount of memory compression for use in processing the query may then be adjusted, based on the estimated execution time and the maximum allowable runtime. The query may then be executed using the adjusted memory compression rate to produce a set of query results, which may then be returned to the requesting entity. | 06-14-2012 |
20120179809 | APPLICATION MONITORING IN A STREAM DATABASE ENVIRONMENT - Methods and systems for monitoring a stream application are disclosed. The stream application is composed from a plurality of processing elements executing on one or more compute nodes. A graphical user interface display presents a user with at least a portion of an operator graph in a running stream application. The operator graph represents the plurality of processing elements, and links between processing elements, corresponding to a flow of data tuples through the stream application. A monitoring application then monitors user interactions with the presentation of the portion of the operator graph on the graphical user interface display and identifies at least a first modification to the stream application based on a processing state of the stream application and the monitored user interactions. | 07-12-2012 |
20120180058 | Configuring An Application For Execution On A Parallel Computer - Methods, systems, and products are disclosed for configuring an application for execution on a parallel computer that include: booting up a first subset of a plurality of nodes in a serial processing mode; booting up a second subset of the plurality of nodes in a parallel processing mode; profiling, prior to application deployment on the parallel computer, the application to identify the serial segments and the parallel segments of the application; and deploying the application for execution on the parallel computer in dependence upon the profile of the application and proximity within the data communications network of the nodes in the first subset relative to the nodes in the second subset. | 07-12-2012 |
20120192193 | Executing An Application On A Parallel Computer - Methods, systems, and products are disclosed for executing an application on a parallel computer having a plurality of nodes. Executing an application on a parallel computer includes: booting up a first subset of a plurality of nodes in a serial processing mode; booting up a second subset of the plurality of nodes in a parallel processing mode; profiling, prior to application execution, an application to identify serial segments of the application, parallel segments of the application, and application data utilized by each of the serial segments and the parallel segments; and executing the application on the plurality of nodes, including migrating, in dependence upon the profile for the application upon encountering the parallel segments during execution, only specific portions of the application and the application data from the nodes booted up in the serial processing mode to the nodes booted up in the parallel processing mode. | 07-26-2012 |
20120203763 | DATABASE QUERY OPTIMIZER THAT TAKES NETWORK CHOICE INTO CONSIDERATION - In a networked computer system that includes multiple nodes and multiple networks interconnecting the nodes, a database query optimizer takes network choice into consideration when optimizing a query. The query optimizer logs historical information for queries that includes network information. A network monitor determines network characteristics of the multiple networks. The query optimizer then selects a network for a query based on the network characteristics, and optionally based on the historical information as well. The result is a query optimizer that takes network choice into consideration, thereby providing a query that is optimized according to the characteristics of the networks. | 08-09-2012 |
20120210330 | Executing A Distributed Java Application On A Plurality Of Compute Nodes - Methods, systems, and products are disclosed for executing a distributed Java application on a plurality of compute nodes. The Java application includes a plurality of jobs distributed among the plurality of compute nodes. The plurality of compute nodes are connected together for data communications through a data communication network. Each of the plurality of compute nodes has installed upon it a Java Virtual Machine (‘JVM’) capable of supporting at least one job of the Java application. Executing a distributed Java application on a plurality of compute nodes includes: tracking, by an application manager, a just-in-time (‘JIT’) compilation history for the JVMs installed on the plurality of compute nodes; and configuring, by the application manager, the plurality of jobs for execution on the plurality of compute nodes in dependence upon the JIT compilation history for the JVMs installed on the plurality of compute nodes. | 08-16-2012 |
20120215764 | ENERGY USAGE AND PERFORMANCE QUERY GOVERNOR - Techniques are described for managing query execution using additional performance capabilities from a set of system resources. Embodiments may receive a query and calculate a first estimated execution time for processing the query using a first amount of performance capabilities from a first set of system resources. If embodiments determine that the first estimated execution time exceeds a threshold amount of time, a second estimated execution time is then calculated for processing the query using a second amount of performance capabilities from the first set of system resources. Here, the second amount of performance capabilities is in addition to the first amount of performance capabilities. If the second estimated execution time does not exceed the threshold amount of time, the query is executed using the second amount of performance capabilities from the first set of system resources. | 08-23-2012 |
20120215934 | ESTIMATING LOAD SHED DATA IN STREAMING DATABASE APPLICATIONS - Techniques are disclosed for processing data streams containing data tuples, where some of the data tuples are load shed. Load shedding refers to a process of selectively discarding some data tuples, e.g., when the amount of received data becomes too overwhelming to manage. When operations are invoked that depend on the values (or count) of the load shed tuples, the value (or counts) of the load shed tuples may be used to perform such operations. For example, and aggregation operation may return the sum, average, mean and/or variance associated with a plurality of tuples processed by a node of the stream application. | 08-23-2012 |
20120216014 | APPLYING ADVANCED ENERGY MANAGER IN A DISTRIBUTED ENVIRONMENT - Techniques are described for abating the negative effects of wait conditions in a distributed system by temporarily decreasing the execution time of processing elements. Embodiments of the invention may generally identify wait conditions from an operator graph and detect the slowest processing element preceding the wait condition based on either historical information or real-time data. Once identified, the slowest processing element may be sped up to lessen the negative consequences of the wait condition. Alternatively, if the slowest processing element shares the same compute node with another processing element in the distributed system, one of the processing elements may be transferred to a different compute node to free additional computing resources on the compute node. | 08-23-2012 |
20120218268 | ANALYSIS OF OPERATOR GRAPH AND DYNAMIC REALLOCATION OF A RESOURCE TO IMPROVE PERFORMANCE - An operator graph analysis mechanism analyzes an operator graph corresponding to an application for problems as the application runs, and determines potential reallocations from a reallocation policy. The reallocation policy may specify potential reallocations depending on whether one or more operators in the operator graph are compute bound, memory bound, communication bound, or storage bound. The operator graph analysis mechanism includes a resource reallocation mechanism that can dynamically change allocation of resources in the system at runtime to address problems detected in the operator graph. The operator graph analysis mechanism thus allows an application represented by an operator graph to dynamically evolve over time to optimize its performance at runtime. | 08-30-2012 |
20120221886 | DISTRIBUTED JOB SCHEDULING IN A MULTI-NODAL ENVIRONMENT - Techniques are described for decentralizing a job scheduler in a distributed system environment. Embodiments of the invention may generally include receiving a job to be performed by a multi-nodal system which includes a cluster of nodes. Instead of a centralized job scheduler assigning the job to a node or nodes, each node has a job scheduler which scans a shared-file system to determine what job to execute on the node. In a job requiring multiple nodes, one of the nodes that joined the multi-nodal job becomes the primary node which then assigns and monitors the job's execution on the multiple nodes. | 08-30-2012 |
20120246509 | GLOBAL DETECTION OF RESOURCE LEAKS IN A MULTI-NODE COMPUTER SYSTEM - A process is disclosed for identifying and recovering from resource leaks on compute nodes of a parallel computing system. A resource monitor stores information about system resources available on a compute node in a clean state. After the compute node runs a job, the resource monitor compares the current resource availability to the clean state. If a resource leak is found, the resource monitor contacts a global resource manger to remove the resource leak. | 09-27-2012 |
20120304058 | DATABASE MONITOR REPLAY - Method, article of manufacture and apparatus for playing back recorded database activity in a graphical user interface. Chronologically ordered database event information is dynamically displayed in a user interface having user-selectable graphical elements associated with predefined playback controls. | 11-29-2012 |
20120310984 | DATA SECURITY FOR A DATABASE IN A MULTI-NODAL ENVIRONMENT - A security mechanism in a database management system enforces processing restrictions stored as metadata to control how different pieces of a multi-nodal application are allowed to access database data to provide data security. The security mechanism preferably checks the data security restrictions for security violations when an execution unit attempts to access the data to insure the nodal conditions are appropriate for access. When the security mechanism determines there is a security violation by a query from an execution unit based on the security restrictions, the security mechanism may send, delay or retry to maintain data security. Nodal conditions herein include time restrictions and relationships with other columns, rows or pieces of information. For example, multiple processing units may be allowed to execute together, but the security mechanism would prohibit these processing units to access specific pieces of information at the same time through the use of metadata in the database. | 12-06-2012 |
20120311172 | OVERLOADING PROCESSING UNITS IN A DISTRIBUTED ENVIRONMENT - Techniques are disclosed for overloading, at one or more nodes, an output of data streams containing data tuples. A first plurality of tuples is received via a first data stream and a second plurality of tuples is received via a second data stream. A first value associated with the first data stream and a second value associated with the second data stream are established based on a specified metric. A third plurality of tuples is output based on the first value and the second value, wherein the third plurality of tuples is a subset of the first plurality of tuples and the second plurality of tuples. | 12-06-2012 |
20130024442 | SYSTEM LOAD QUERY GOVERNOR - Techniques are disclosed for query processing. In one embodiment, a query is received for execution in a processing environment. Based on a measure of current load of the processing environment, a maximum amount of a resource that the query is allowed to consume is determined. An amount of the resource that the query is to consume is estimated. Execution of the query is managed based on a comparison between the maximum amount and the estimated amount. | 01-24-2013 |
20130031124 | MANAGEMENT SYSTEM FOR PROCESSING STREAMING DATA - Techniques are disclosed for evaluating tuples for processing by a stream application having a plurality of process elements. In one embodiment, at least one tuple to be processed by at least one processing element of the stream application is identified. A maximum duration for which the at least one processing element is allowed to process the at least one tuple is determined. A duration for which the at least one processing element is likely to process the at least one tuple is also estimated. Processing of the at least one tuple is managed based on a comparison between the maximum duration and the estimated duration. | 01-31-2013 |
20130031263 | DYNAMIC RUNTIME CHOOSING OF PROCESSING COMMUNICATION METHODS - Techniques are described for assigning and changing communication protocols for a pair of processing elements. The communication protocol determines how the pair of processing elements transmits data in a stream application. The pair may be assigned a communication protocol (e.g., TCP/IP or a protocol that uses a relational database, shared file system, or shared memory) before the operator graph begins to stream data. This assignment may be based on a priority of the processing elements and/or a priority of the communication protocols. After the operator graph begins to stream data, the pair of processing elements may switch to a different communication protocol. The decision to switch the communication protocol may be based on whether the pair of processing elements or assigned communication protocol is meeting established performance standards for the stream application. | 01-31-2013 |
20130031335 | USING PREDICTIVE DETERMINISM WITHIN A STREAMING ENVIRONMENT - Techniques are described for transmitting predicted output data on a processing element in a stream computing application instead of processing currently received input data. The stream computing application monitors the output of a processing element and determines whether its output is predictable, for example, if the previously transmitted output values are within a predefined range or if one or more input values correlate with the same one or more output values. The application may then generate a predicted output value to transmit from the processing element instead of transmitting a processed output value based on current input values. The predicted output value may be, for example, an average of the previously transmitted output values or a previously transmitted output value that was transmitted in response to a previously received input value that is similar to a currently received input value. Moreover, the processing element or elements that transmit the predicted output data may be upstream from the processing element with the predictable output. | 01-31-2013 |
20130031556 | DYNAMIC REDUCTION OF STREAM BACKPRESSURE - Techniques are described for eliminating backpressure in a distributed system by changing the rate data flows through a processing element. Backpressure occurs when data throughput in a processing element begins to decrease, for example, if new processing elements are added to the operating chart or if the distributed system is required to process more data. Indicators of backpressure (current or future) may be monitored. Once current backpressure or potential backpressure is identified, the operator graph or data rates may be altered to alleviate the backpressure. For example, a processing element may reduce the data rates it sends to processing elements that are downstream in the operator graph, or processing elements and/or data paths may be eliminated. In one embodiment, processing elements and associate data paths may be prioritized so that more important execution paths are maintained. In another embodiment, if a request to add one or more processing elements may cause future backpressure, the request may be refused. | 01-31-2013 |
20130054538 | INVALIDATING STREAMS IN AN OPERATOR GRAPH - Techniques are disclosed for invalidating, at one or more processing elements, data streams containing data tuples. A plurality of tuples is received via a data stream, whereupon the data stream is determined to be invalid based on at least one tuple in the plurality of tuples. The data stream is then invalidated, and a message is issued that causes one or more data streams included in the stream-based computing system and related to the invalidated data stream to also be invalidated. | 02-28-2013 |
20130054779 | STREAM APPLICATION PERFORMANCE MONITORING METRICS - Techniques are disclosed for calculating performance metrics associated with a data stream. A processing element configured to process data tuples flowing through a stream-based computing system receives data tuples via the data stream. Each of the one or more tuples is processed at the processing element, and each of the one or more tuples are associated with metadata that includes information related to the processing of the tuple by the processing element. Performance metrics are then calculated for the data stream based on the metadata associated with the one or more tuples. | 02-28-2013 |
20130074078 | CALL STACK AGGREGATION AND DISPLAY - A call stack aggregation mechanism aggregates call stacks from multiple threads of execution and displays the aggregated call stack to a user in a manner that visually distinguishes between the different call stacks in the aggregated call stack. The multiple threads of execution may be on the same computer system or on separate computer systems. | 03-21-2013 |
20130074146 | DATA SECURITY FOR A DATABASE IN A MULTI-NODAL ENVIRONMENT - A security mechanism in a database management system enforces processing restrictions stored as metadata to control how different pieces of a multi-nodal application are allowed to access database data to provide data security. The security mechanism preferably checks the data security restrictions for security violations when an execution unit attempts to access the data to insure the nodal conditions are appropriate for access. When the security mechanism determines there is a security violation by a query from an execution unit based on the security restrictions, the security mechanism may send, delay or retry to maintain data security. Nodal conditions herein include time restrictions and relationships with other columns, rows or pieces of information. For example, multiple processing units may execute together, but the security mechanism would prohibit these processing units to access specific pieces of information at the same time through the use of metadata in the database. | 03-21-2013 |
20130074192 | DATA SECURITY IN A MULTI-NODAL ENVIRONMENT - A data security manager in a multi-nodal environment enforces processing constraints stored as security relationships that control how different pieces of a multi-nodal application (called execution units) are allowed to execute to insure data security. The security manager preferably checks the security relationships for security violations when new execution units start execution, when data moves to or from an execution unit, and when an execution unit requests external services. Where the security manager determines there is a security violation based on the security relationships, the security manager may move, delay or kill an execution unit to maintain data security. | 03-21-2013 |
20130080400 | SPECULATIVE EXECUTION IN A REAL-TIME DATA ENVIRONMENT - Techniques are described for speculatively executing operations on data in a data stream in parallel in a manner that increases the efficiency of the stream-based application. In addition to executing operations in parallel, embodiments of the invention may determine whether certain results produced by the parallel operations are valid results and discard any results determined to be invalid. | 03-28-2013 |
20130080410 | ACTIVE MEMORY EXPANSION IN A DATABASE ENVIRONMENT TO QUERY NEEDED/UNEEDED RESULTS - Techniques are described for estimating and managing memory compression for request processing. Embodiments of the invention may generally include receiving a request for data, determining if the requested data contains any compressed data, and sending the requesting entity only the uncompressed data. A separate embodiment generally includes receiving a request for data, determining if the requested data contains any compressed data, gathering uncompression criteria about the requested data, and using the uncompression criteria to selectively determine what portion of the compressed data to uncompress. | 03-28-2013 |
20130080600 | MANAGEMENT SYSTEM FOR PROCESSING STREAMING DATA - Techniques are disclosed for evaluating tuples for processing by a stream application having a plurality of process elements. In one embodiment, at least one tuple to be processed by at least one processing element of the stream application is identified. A maximum duration for which the at least one processing element is allowed to process the at least one tuple is determined. A duration for which the at least one processing element is likely to process the at least one tuple is also estimated. Processing of the at least one tuple is managed based on a comparison between the maximum duration and the estimated duration. | 03-28-2013 |
20130080652 | DYNAMIC RUNTIME CHOOSING OF PROCESSING COMMUNICATION METHODS - Techniques are described for assigning and changing communication protocols for a pair of processing elements. The communication protocol determines how the pair of processing elements transmits data in a stream application. The pair may be assigned a communication protocol (e.g., TCP/IP or a protocol that uses a relational database, shared file system, or shared memory) before the operator graph begins to stream data. This assignment may be based on a priority of the processing elements and/or a priority of the communication protocols. After the operator graph begins to stream data, the pair of processing elements may switch to a different communication protocol. The decision to switch the communication protocol may be based on whether the pair of processing elements or assigned communication protocol is meeting established performance standards for the stream application. | 03-28-2013 |
20130080653 | USING PREDICTIVE DETERMINISM WITHIN A STREAMING ENVIRONMENT - Techniques are described for transmitting predicted output data on a processing element in a stream computing application instead of processing currently received input data. The stream computing application monitors the output of a processing element and determines whether its output is predictable, for example, if the previously transmitted output values are within a predefined range or if one or more input values correlate with the same one or more output values. The application may then generate a predicted output value to transmit from the processing element instead of transmitting a processed output value based on current input values. The predicted output value may be, for example, an average of the previously transmitted output values or a previously transmitted output value that was transmitted in response to a previously received input value that is similar to a currently received input value. | 03-28-2013 |
20130080654 | OVERLOADING PROCESSING UNITS IN A DISTRIBUTED ENVIRONMENT - Techniques are disclosed for overloading, at one or more nodes, an output of data streams containing data tuples. A first plurality of tuples is received via a first data stream and a second plurality of tuples is received via a second data stream. A first value associated with the first data stream and a second value associated with the second data stream are established based on a specified metric. A third plurality of tuples is output based on the first value and the second value, wherein the third plurality of tuples is a subset of the first plurality of tuples and the second plurality of tuples. | 03-28-2013 |
20130080655 | ESTIMATING LOAD SHED DATA IN STREAMING DATABASE APPLICATIONS - Techniques are disclosed for processing data streams containing data tuples, where some of the data tuples are load shed. Load shedding refers to a process of selectively discarding some data tuples, e.g., when the amount of received data becomes too overwhelming to manage. When operations are invoked that depend on the values (or count) of the load shed tuples, the value (or counts) of the load shed tuples may be used to perform such operations. For example, and aggregation operation may return the sum, average, mean and/or variance associated with a plurality of tuples processed by a node of the stream application. | 03-28-2013 |
20130080824 | DISTRIBUTED JOB SCHEDULING IN A MULTI-NODAL ENVIRONMENT - Techniques are described for decentralizing a job scheduler in a distributed system environment. Embodiments of the invention may generally include receiving a job to be performed by a multi-nodal system which includes a cluster of nodes. Instead of a centralized job scheduler assigning the job to a node or nodes, each node has a job scheduler which scans a shared-file system to determine what job to execute on the node. In a job requiring multiple nodes, one of the nodes that joined the multi-nodal job becomes the primary node which then assigns and monitors the job's execution on the multiple nodes. | 03-28-2013 |
20130081042 | DYNAMIC REDUCTION OF STREAM BACKPRESSURE - Techniques are described for eliminating backpressure in a distributed system by changing the rate data flows through a processing element. Backpressure occurs when data throughput in a processing element begins to decrease, for example, if new processing elements are added to the operating chart or if the distributed system is required to process more data. Indicators of backpressure (current or future) may be monitored. Once current backpressure or potential backpressure is identified, the operator graph or data rates may be altered to alleviate the backpressure. For example, a processing element may reduce the data rates it sends to processing elements that are downstream in the operator graph, or processing elements and/or data paths may be eliminated. In one embodiment, processing elements and associate data paths may be prioritized so that more important execution paths are maintained. | 03-28-2013 |
20130081046 | ANALYSIS OF OPERATOR GRAPH AND DYNAMIC REALLOCATION OF A RESOURCE TO IMPROVE PERFORMANCE - An operator graph analysis mechanism analyzes an operator graph corresponding to an application for problems as the application runs, and determines potential reallocations from a reallocation policy. The reallocation policy may specify potential reallocations depending on whether one or more operators in the operator graph are compute bound, memory bound, communication bound, or storage bound. The operator graph analysis mechanism includes a resource reallocation mechanism that can dynamically change allocation of resources in the system at runtime to address problems detected in the operator graph. The operator graph analysis mechanism thus allows an application represented by an operator graph to dynamically evolve over time to optimize its performance at runtime. | 03-28-2013 |
20130091496 | OPTIMIZING JUST-IN-TIME COMPILING FOR A JAVA APPLICATION EXECUTING ON A COMPUTE NODE - Methods, systems, and products are disclosed for optimizing just-in-time (‘JIT’) compiling for a Java application executing on a compute node, the compute node having installed upon it a Java Virtual Machine (‘JVM’) capable of supporting the Java application, that include: identifying, by an application manager, a particular portion of the Java application; assigning, by the application manager, a JIT level to the particular portion of the Java application; and jitting, by the JVM installed on the compute node, the particular portion of the Java application in dependence upon the JIT level assigned to that particular portion of the Java application. | 04-11-2013 |
20130097323 | DYNAMIC PROCESSING UNIT RELOCATION IN A MULTI-NODAL ENVIRONMENT BASED ON INCOMING PHYSICAL DATA - A relocation mechanism in a multi-nodal computer environment dynamically routes processing units in a distributed computer system based on incoming physical data into the processing unit. The relocation mechanism makes an initial location decision to place a processing unit onto a node in the distributed computer system. The relocation mechanism monitors physical data flowing into a processing unit or node and dynamically relocates the processing unit to another type of node within the ‘cloud’ of nodes based on the type of physical data or pattern of data flowing into the processing unit. The relocation mechanism may use one or more rules with criteria for different data types observed in the data flow to optimize when to relocate the processing units. | 04-18-2013 |
20130097612 | DYNAMIC RUN TIME ALLOCATION OF DISTRIBUTED JOBS WITH APPLICATION SPECIFIC METRICS - A job optimizer dynamically changes the allocation of processing units on a multi-nodal computer system. A distributed application is organized as a set of connected processing units. The arrangement of the processing units is dynamically changed at run time to optimize system resources and interprocess communication. A collector collects application specific metrics determined by application plug-ins. A job optimizer analyzes the collected metrics and determines how to dynamically arrange the processing units within the jobs. The job optimizer may determine to combine multiple processing units into a job on a single node when there is an overutilization of an interprocess communication between processing units. Alternatively, the job optimizer may determine to split a job's processing units into multiple jobs on different nodes where one or more of the processing units are over utilizing the resources on the node. | 04-18-2013 |
20130103832 | STREAM APPLICATION PERFORMANCE MONITORING METRICS - Techniques are disclosed for calculating performance metrics associated with a data stream. A processing element configured to process data tuples flowing through a stream-based computing system receives data tuples via the data stream. Each of the one or more tuples is processed at the processing element, and each of the one or more tuples are associated with metadata that includes information related to the processing of the tuple by the processing element. Performance metrics are then calculated for the data stream based on the metadata associated with the one or more tuples. | 04-25-2013 |
20130104143 | RUN-TIME ALLOCATION OF FUNCTIONS TO A HARDWARE ACCELERATOR - An accelerator work allocation mechanism determines at run-time which functions to allocate to a hardware accelerator based on a defined accelerator policy, and based on an analysis performed at run-time. The analysis includes reading the accelerator policy, and determining whether a particular function satisfies the accelerator policy. If so, the function is allocated to the hardware accelerator. If not, the function is allocated to the processor. | 04-25-2013 |
20130124446 | DETECTING DISALLOWED COMBINATIONS OF DATA WITHIN A PROCESSING ELEMENT - Techniques are described for detecting disallowed combinations of data within a processing element. Embodiments of the invention may generally receive data to be processed using the processing element and determine whether the received data and a current working state violate one or more rules describing disallowed combinations of data. If a disallowed combination is detected, embodiments of the invention may handle the processing of the received data in an alternate way that prevents disallowed combinations of data within the processing element. | 05-16-2013 |
20130124599 | DYNAMIC RESOURCE ADJUSTMENT FOR A DISTRIBUTED PROCESS ON A MULTI-NODE COMPUTER SYSTEM - A method dynamically adjusts the resources available to a processing unit of a distributed computer process executing on a multi-node computer system. The resources for the processing unit are adjusted based on the data other processing units handle or the execution path of code in an upstream or downstream processing unit in the distributed process or application. | 05-16-2013 |
20130124726 | DYNAMIC RESOURCE ADJUSTMENT FOR A DISTRIBUTED PROCESS ON A MULTI-NODE COMPUTER SYSTEM - A method dynamically adjusts the resources available to a processing unit of a distributed computer process executing on a multi-node computer system. The resources for the processing unit are adjusted based on the data other processing units handle or the execution path of code in an upstream or downstream processing unit in the distributed process or application. | 05-16-2013 |
20130139174 | DYNAMIC RUN TIME ALLOCATION OF DISTRIBUTED JOBS - A job optimizer dynamically changes the allocation of processing units on a multi-nodal computer system. A distributed application is organized as a set of connected processing units. The arrangement of the processing units is dynamically changed at run time to optimize system resources and interprocess communication. A collector collects metrics of the system, nodes, application, jobs and processing units that will be used to determine how to best allocate the jobs on the system. A job optimizer analyzes the collected metrics to dynamically arrange the processing units within the jobs. The job optimizer may determine to combine multiple processing units into a job on a single node when there is an overutilization of interprocess communication between processing units. Alternatively, the job optimizer may determine to split a job's processing units into multiple jobs on different nodes where the processing units are over utilizing the resources on the node. | 05-30-2013 |
20130159284 | AUTONOMIC REFRESH OF A MATERIALIZED QUERY TABLE IN A COMPUTER DATABASE - A method autonomically refreshes a materialized query table (MQT) in a computer database to improve database performance and utility. In preferred embodiments, the query optimizer autonomically initiates a refresh of MQT depending on an estimated time for the query to access the base tables. In other preferred embodiments, the query optimizer estimates the time for the query to access the base tables and compares it to the estimated time to refresh the MQT to determine whether to refresh the MQT and run the query over the MQT rather than the base tables. | 06-20-2013 |
20130159287 | DATABASE QUERY OPTIMIZER THAT TAKES NETWORK CHOICE INTO CONSIDERATION - In a networked computer system that includes multiple nodes and multiple networks interconnecting the nodes, a database query optimizer takes network choice into consideration when optimizing a query. The query optimizer logs historical information for queries that includes network information. A network monitor determines network characteristics of the multiple networks. The query optimizer then selects a network for a query based on the network characteristics, and optionally based on the historical information as well. The result is a query optimizer that takes network choice into consideration, thereby providing a query that is optimized according to the characteristics of the networks. | 06-20-2013 |
20130159745 | OPTIMIZING POWER CONSUMPTION AND PERFORMANCE IN A HYBRID COMPUTER ENVIRONMENT - A method for optimizing efficiency and power consumption in a hybrid computer system is disclosed. The hybrid computer system may comprise one or more front-end nodes connected to a multi-node computer system. Portions of an application may be offloaded from the front-end nodes to the multi-node computer system. By building historical profiles of the applications running on the multi-node computer system, the system can analyze the trade offs between power consumption and performance. For example, if running the application on the multi-node computer system cuts the run time by 5% but increases power consumption by 20% it may be more advantageous to simply run the entire application on the front-end. | 06-20-2013 |
20130159983 | OPTIMIZING JUST-IN-TIME COMPILING FOR A JAVA APPLICATION EXECUTING ON A COMPUTE NODE - Methods, systems, and products are disclosed for optimizing just-in-time (‘JIT’) compiling for a Java application executing on a compute node, the compute node having installed upon it a Java Virtual Machine (‘JVM’) capable of supporting the Java application, that include: identifying, by an application manager, a particular portion of the Java application; assigning, by the application manager, a JIT level to the particular portion of the Java application; and jitting, by the JVM installed on the compute node, the particular portion of the Java application in dependence upon the JIT level assigned to that particular portion of the Java application. | 06-20-2013 |
20130160136 | DATA SECURITY IN A MULTI-NODAL ENVIRONMENT - A data security manager in a multi-nodal environment enforces processing constraints stored as security relationships that control how different pieces of a multi-nodal application (called execution units) are allowed to execute to insure data security. The security manager preferably checks the security relationships for security violations when new execution units start execution, when data moves to or from an execution unit, and when an execution unit requests external services. Where the security manager determines there is a security violation based on the security relationships, the security manager may move, delay or kill an execution unit to maintain data security. | 06-20-2013 |
20130166532 | STRING SEARCHES IN A COMPUTER DATABASE - A query optimizer improves string searches in a computer database that sequentially search for a string in a database record. The query optimizer optimizes the query to search records of a database from a specified start position other than the beginning of the record. The specified start position of the search may be determined by from historical information stored from previous searches. Alternatively, the query optimizer determines the specified start position of the search based on an overriding starting position provided by a system administrator. The query optimizer may also direct that the database record be reorganized to more efficiently search for strings in the record. | 06-27-2013 |
20130166533 | STRING SEARCHES IN A COMPUTER DATABASE - A query optimizer improves string searches in a computer database that sequentially search for a string in a database record. The query optimizer optimizes the query to search records of a database from a specified start position other than the beginning of the record. The specified start position of the search may be determined by from historical information stored from previous searches. Alternatively, the query optimizer determines the specified start position of the search based on an overriding starting position provided by a system administrator. The query optimizer may also direct that the database record be reorganized to more efficiently search for strings in the record. | 06-27-2013 |
20130166617 | ENHANCED BARRIER OPERATOR WITHIN A STREAMING ENVIRONMENT - Techniques are described for processing data. Embodiments receive streaming data to be processed by a plurality of processing elements. An operator graph of the plurality of processing elements that defines at least one execution path is established. Additionally, a first processing element in the operator graph includes a barrier operator that joins the output of one or more upstream operators included in one or more of the plurality of processing elements. Embodiments initiate one or more timeout conditions at the barrier operator. Embodiments also determine, at the first processing element, that one or more timeout conditions have been satisfied before data has been received from each of the one or more upstream operators. Upon determining that the one or more timeout conditions have been satisfied, Embodiments generate output data at the barrier operator without the data from at least one of the one or more upstream operators. | 06-27-2013 |
20130166618 | PREDICTIVE OPERATOR GRAPH ELEMENT PROCESSING - Techniques are described for predictively starting a processing element. Embodiments receive streaming data to be processed by a plurality of processing elements. An operator graph of the plurality of processing elements that defines at least one execution path is established. Embodiments determine a historical startup time for a first processing element in the operator graph, where, once started, the first processing element begins normal operations once the first processing element has received a requisite amount of data from one or more upstream processing elements. Additionally, embodiments determine an amount of time the first processing element takes to receive the requisite amount of data from the one or more upstream processing elements. The first processing element is then predictively started at a first startup time based on the determined historical startup time and the determined amount of time historically taken to receive the requisite amount of data. | 06-27-2013 |
20130166620 | ENHANCED BARRIER OPERATOR WITHIN A STREAMING ENVIRONMENT - Techniques are described for processing data. Embodiments receive streaming data to be processed by a plurality of processing elements. An operator graph of the plurality of processing elements that defines at least one execution path is established. Additionally, a first processing element in the operator graph includes a barrier operator that joins the output of one or more upstream operators included in one or more of the plurality of processing elements. Embodiments initiate one or more timeout conditions at the barrier operator. Embodiments also determine, at the first processing element, that one or more timeout conditions have been satisfied before data has been received from each of the one or more upstream operators. Upon determining that the one or more timeout conditions have been satisfied, Embodiments generate output data at the barrier operator without the data from at least one of the one or more upstream operators. | 06-27-2013 |
20130166888 | PREDICTIVE OPERATOR GRAPH ELEMENT PROCESSING - Techniques are described for predictively starting a processing element. Embodiments receive streaming data to be processed by a plurality of processing elements. An operator graph of the plurality of processing elements that defines at least one execution path is established. Embodiments determine a historical startup time for a first processing element in the operator graph, where, once started, the first processing element begins normal operations once the first processing element has received a requisite amount of data from one or more upstream processing elements. Additionally, embodiments determine an amount of time the first processing element takes to receive the requisite amount of data from the one or more upstream processing elements. The first processing element is then predictively started at a first startup time based on the determined historical startup time and the determined amount of time historically taken to receive the requisite amount of data. | 06-27-2013 |
20130166942 | UNFUSING A FAILING PART OF AN OPERATOR GRAPH - Techniques for managing a fused processing element are described. Embodiments receive streaming data to be processed by a plurality of processing elements. Additionally, an operator graph of the plurality of processing elements is established. The operator graph defines at least one execution path and wherein at least one of the processing elements of the operator graph is configured to receive data from at least one upstream processing element and transmit data to at least one downstream processing element. Embodiments detect an error condition has been satisfied at a first one of the plurality of processing elements, wherein the first processing element contains a plurality of fused operators. At least one of the plurality of fused operators is selected for removal from the first processing element. Embodiments then remove the selected at least one fused operator from the first processing element. | 06-27-2013 |
20130166948 | UNFUSING A FAILING PART OF AN OPERATOR GRAPH - Techniques for managing a fused processing element are described. Embodiments receive streaming data to be processed by a plurality of processing elements. Additionally, an operator graph of the plurality of processing elements is established. The operator graph defines at least one execution path and wherein at least one of the processing elements of the operator graph is configured to receive data from at least one upstream processing element and transmit data to at least one downstream processing element. Embodiments detect an error condition has been satisfied at a first one of the plurality of processing elements, wherein the first processing element contains a plurality of fused operators. At least one of the plurality of fused operators is selected for removal from the first processing element. Embodiments then remove the selected at least one fused operator from the first processing element. | 06-27-2013 |
20130166961 | DETECTING AND RESOLVING ERRORS WITHIN AN APPLICATION - Techniques for managing errors within an application are provided. Embodiments monitor errors occurring in each of a plurality of portions of the application while the application is executing. An error occurring in a first one of the plurality of portions of the application is detected. Additionally, upon detecting the error occurring in the first portion, embodiments determine whether to prevent subsequent executions of the first portion of the application. | 06-27-2013 |
20130166962 | DETECTING AND RESOLVING ERRORS WITHIN AN APPLICATION - Techniques for managing errors within an application are provided. Embodiments monitor errors occurring in each of a plurality of portions of the application while the application is executing. An error occurring in a first one of the plurality of portions of the application is detected. Additionally, upon detecting the error occurring in the first portion, embodiments determine whether to prevent subsequent executions of the first portion of the application. | 06-27-2013 |
20130173543 | MATERIALIZED QUERY TABLE JOURNALING IN A COMPUTER DATABASE SYSTEM - An apparatus and method utilize MQTs in a more efficient manner in a high availability computer database to improve database performance and utility. In preferred embodiments, an MQT control file indicates whether journal entries for specific tables are to be propagated to replicated databases residing on other computer servers. In other embodiments, the MQT control file includes metrics that are used to control when the propagation is turned on and off. | 07-04-2013 |
20130173544 | MATERIALIZED QUERY TABLE JOURNALING IN A COMPUTER DATABASE SYSTEM - An apparatus and method utilize MQTs in a more efficient manner in a high availability computer database to improve database performance and utility. In preferred embodiments, an MQT control file indicates whether journal entries for specific tables are to be propagated to replicated databases residing on other computer servers. In other embodiments, the MQT control file includes metrics that are used to control when the propagation is turned on and off. | 07-04-2013 |
20130179432 | QUERY EXECUTION AND OPTIMIZATION WITH AUTONOMIC ERROR RECOVERY FROM NETWORK FAILURES IN A PARALLEL COMPUTER SYSTEM WITH MULTIPLE NETWORKS - A database query execution monitor determines if an network error or low performance condition exists and then where possible modifies the query. The query execution monitor then determines an alternate query execution plan to continue execution of the query. The query optimizer can re-optimize the query to use a different network or node. Thus, the query execution monitor allows autonomic error recovery for network failures using an alternate query execution. The alternate query execution could also be determined at the initial optimization time and then this alternate plan used to execute a query in the case of a particular network failure. | 07-11-2013 |
20130179585 | TRIGGERING WINDOW CONDITIONS BY STREAMING FEATURES OF AN OPERATOR GRAPH - In a stream computing application, data may be transmitted between operators using tuples. However, the receiving operator may not evaluate these tuples as they arrive but instead wait to evaluate a group of tuples—i.e., a window. A window is typically triggered when a buffer associated with the receiving operator reaches a maximum window size or when a predetermined time period has expired. Additionally, a window may be triggered by a monitoring a tuple rate—i.e., the rate at which the operator receives the tuples. If the tuple rate exceeds or falls below a threshold, a window may be triggered. Further, the number of exceptions, or the rate at which an operator throws exceptions, may be monitored. If either of these parameters satisfies a threshold, a window may be triggered, thereby instructing an operator to evaluate the tuples contained within the window. | 07-11-2013 |
20130179586 | TRIGGERING WINDOW CONDITIONS USING EXCEPTION HANDLING - In a stream computing application, data may be transmitted between operators using tuples. However, the receiving operator may not evaluate these tuples as they arrive but instead wait to evaluate a group of tuples—i.e., a window. A window is typically triggered when a buffer associated with the receiving operator reaches a maximum window size or when a predetermined time period has expired. Additionally, a window may be triggered by a monitoring a tuple rate—i.e., the rate at which the operator receives the tuples. If the tuple rate exceeds or falls below a threshold, a window may be triggered. Further, the number of exceptions, or the rate at which an operator throws exceptions, may be monitored. If either of these parameters satisfies a threshold, a window may be triggered, thereby instructing an operator to evaluate the tuples contained within the window. | 07-11-2013 |
20130179591 | TRIGGERING WINDOW CONDITIONS BY STREAMING FEATURES OF AN OPERATOR GRAPH - In a stream computing application, data may be transmitted between operators using tuples. However, the receiving operator may not evaluate these tuples as they arrive but instead wait to evaluate a group of tuples—i.e., a window. A window is typically triggered when a buffer associated with the receiving operator reaches a maximum window size or when a predetermined time period has expired. Additionally, a window may be triggered by a monitoring a tuple rate—i.e., the rate at which the operator receives the tuples. If the tuple rate exceeds or falls below a threshold, a window may be triggered. Further, the number of exceptions, or the rate at which an operator throws exceptions, may be monitored. If either of these parameters satisfies a threshold, a window may be triggered, thereby instructing an operator to evaluate the tuples contained within the window. | 07-11-2013 |
20130185278 | QUERY OPTIMIZATION IN A PARALLEL COMPUTER SYSTEM TO REDUCE NETWORK TRAFFIC - A database query optimizer optimizes a query that uses multiple networks. The query optimizer optimizes a query to reduce network traffic on a network or node that is overloaded or above an established parameter in a node/network attribute table. The query optimization to reduce network traffic may result in a sub-optimal query in other respects such as execution time. The result is a query optimizer that rewrites or optimizes a query to execute on multiple nodes or networks to reduce traffic on a network or node according to the loading characteristics and assigned attributes of a node or network. | 07-18-2013 |
20130185279 | QUERY OPTIMIZATION IN A PARALLEL COMPUTER SYSTEM WITH MULTIPLE NETWORKS - A database query optimizer optimizes a query that uses multiple networks. The database query optimizer optimizes a query that uses multiple networks to satisfy the query by splitting the query execution to use multiple networks. Thus, the query optimizer rewrites or optimizes a query to execute on multiple nodes or networks to more efficiently execute the query and reduce network traffic on a network. The query optimizer uses plan cache statistics to determine whether to use multiple networks to optimize the query. | 07-18-2013 |
20130185283 | QUERY OPTIMIZATION IN A PARALLEL COMPUTER SYSTEM WITH MULTIPLE NETWORKS - A database query optimizer optimizes a query that uses multiple networks. The database query optimizer optimizes a query that uses multiple networks to satisfy the query by splitting the query execution to use multiple networks. Thus, the query optimizer rewrites or optimizes a query to execute on multiple nodes or networks to more efficiently execute the query and reduce network traffic on a network. The query optimizer uses plan cache statistics to determine whether to use multiple networks to optimize the query. | 07-18-2013 |
20130185301 | INSERTING DATA INTO AN IN-MEMORY DISTRIBUTED NODAL DATABASE - A database loader loads data to an in-memory database across multiple nodes in a parallel computing system. The database loader uses SQL flags, historical information gained from monitoring prior query execution times and patterns, and node and network configuration to determine how to effectively cluster data attributes across multiple nodes. The database loader may also allow a system administrator to force placement of database structures in particular nodes. | 07-18-2013 |
20130185302 | INSERTING DATA INTO AN IN-MEMORY DISTRIBUTED NODAL DATABASE - A database loader loads data to an in-memory database across multiple nodes in a parallel computing system. The database loader uses SQL flags, historical information gained from monitoring prior query execution times and patterns, and node and network configuration to determine how to effectively cluster data attributes across multiple nodes. The database loader may also allow a system administrator to force placement of database structures in particular nodes. | 07-18-2013 |
20130185423 | DYNAMIC DISTRIBUTION OF NODES ON A MULTI-NODE COMPUTER SYSTEM - I/O nodes are dynamically distributed on a multi-node computing system. An I/O configuration mechanism located in the service node of a multi-node computer system controls the distribution of the I/O nodes. The I/O configuration mechanism uses job information located in a job record to initially configure the I/O node distribution. The I/O configuration mechanism further monitors the I/O performance of the executing job to then dynamically adjusts the I/O node distribution based on the I/O performance of the executing job. | 07-18-2013 |
20130185588 | QUERY EXECUTION AND OPTIMIZATION WITH AUTONOMIC ERROR RECOVERY FROM NETWORK FAILURES IN A PARALLEL COMPUTER SYSTEM WITH MULTIPLE NETWORKS - A database query execution monitor determines if a network error or low performance condition exists and then where possible modifies the query. The query execution monitor then determines an alternate query execution plan to continue execution of the query. The query optimizer can re-optimize the query to use a different network or node. Thus, the query execution monitor allows autonomic error recovery for network failures using an alternate query execution. The alternate query execution could also be determined at the initial optimization time and then this alternate plan used to execute a query in the case of a particular network failure. | 07-18-2013 |
20130185731 | DYNAMIC DISTRIBUTION OF NODES ON A MULTI-NODE COMPUTER SYSTEM - I/O nodes are dynamically distributed on a multi-node computing system. An I/O configuration mechanism located in the service node of a multi-node computer system controls the distribution of the I/O nodes. The I/O configuration mechanism uses job information located in a job record to initially configure the I/O node distribution. The I/O configuration mechanism further monitors the I/O performance of the executing job to then dynamically adjusts the I/O node distribution based on the I/O performance of the executing job. | 07-18-2013 |
20130191352 | DYNAMIC PARTIAL UNCOMPRESSION OF A DATABASE TABLE - A database dynamic partial uncompression mechanism determines when to dynamically uncompress one or more compressed portions of a database table that also includes uncompressed portions. A query may include an express term that specifies whether or not to skip compressed portions. In addition, a query may include associated information that specifies whether or not to skip compressed portions, and one or more thresholds that may be used to determine if the system is too busy to perform uncompression. A display mechanism may also determine whether or not to display compressed portions. The uncompression may occur at the database server or at a client. The database dynamic partial uncompression mechanism thus performs dynamic uncompression in a way that preferably uncompresses one or more compressed portions of a partially compressed database table only when the compressed portions satisfy a query and/or need to be displayed. | 07-25-2013 |
20130191353 | DYNAMIC PARTIAL UNCOMPRESSION OF A DATABASE TABLE - A database dynamic partial uncompression mechanism determines when to dynamically uncompress one or more compressed portions of a database table that also includes uncompressed portions. A query may include an express term that specifies whether or not to skip compressed portions. In addition, a query may include associated information that specifies whether or not to skip compressed portions, and one or more thresholds that may be used to determine if the system is too busy to perform uncompression. A display mechanism may also determine whether or not to display compressed portions. The uncompression may occur at the database server or at a client. The database dynamic partial uncompression mechanism thus performs dynamic uncompression in a way that preferably uncompresses one or more compressed portions of a partially compressed database table only when the compressed portions satisfy a query and/or need to be displayed. | 07-25-2013 |
20130191369 | QUERY OPTIMIZATION IN A PARALLEL COMPUTER SYSTEM TO REDUCE NETWORK TRAFFIC - A database query optimizer optimizes a query that uses multiple networks. The query optimizer optimizes a query to reduce network traffic on a network or node that is overloaded or above an established parameter in a node/network attribute table. The query optimization to reduce network traffic may result in a sub-optimal query in other respects such as execution time. The result is a query optimizer that rewrites or optimizes a query to execute on multiple nodes or networks to reduce traffic on a network or node according to the loading characteristics and assigned attributes of a node or network. | 07-25-2013 |
20130191504 | OPTIMIZED PEER-TO-PEER FILE TRANSFERS ON A MULTI-NODE COMPUTER SYSTEM - A method and apparatus performs peer-to-peer file transfers on a High Performance Computing (HPC) cluster such as a Beowulf cluster. A peer-to-peer file tracker (PPFT) allows operating system, application and data files to be moved from a pre-loaded node to another node of the HPC cluster. A peer-to-peer (PTP) client is loaded into the nodes to facilitate PTP file transfers to reduce loading on networks, network switches and file servers to reduce the time needed to load the nodes with these files to increase overall efficiency of the multi-node computing system. The selection of the nodes participating in file transfers can be based on network topology, network utilization, job status and predicted network/computer utilization. This selection can be dynamic, changing during the file transfers as resource conditions change. The policies used to choose resources can be configured by an administrator. | 07-25-2013 |
20130191840 | RESOURCE ALLOCATION BASED ON ANTICIPATED RESOURCE UNDERUTILIZATION IN A LOGICALLY PARTITIONED MULTI-PROCESSOR ENVIRONMENT - A method, apparatus and program product for allocating resources in a logically partitioned multiprocessor environment. Resource usage is monitored in a first logical partition in the logically partitioned multiprocessor environment to predict a future underutilization of a resource in the first logical partition. An application executing in a second logical partition in the logically partitioned multiprocessor environment is configured for execution in the second logical partition with an assumption made that at least a portion of the underutilized resource is allocated to the second logical partition during at least a portion of the predicted future underutilization of the resource. | 07-25-2013 |
20130198318 | PROCESSING ELEMENT MANAGEMENT IN A STREAMING DATA SYSTEM - Stream applications may inefficiently use the hardware resources that execute the processing elements of the data stream. For example, a compute node may host four processing elements and execute each using a CPU. However, other CPUs on the compute node may sit idle. To take advantage of these available hardware resources, a stream programmer may identify one or more processing elements that may be cloned. The cloned processing elements may be used to generate a different execution path that is parallel to the execution path that includes the original processing elements. Because the cloned processing elements contain the same operators as the original processing elements, the data stream that was previously flowing through only the original processing element may be split and sent through both the original and cloned processing elements. In this manner, the parallel execution path may use underutilized hardware resources to increase the throughput of the data stream. | 08-01-2013 |
20130198343 | OPTIMIZED PEER-TO-PEER FILE TRANSFERS ON A MULTI-NODE COMPUTER SYSTEM - A method and apparatus performs peer-to-peer file transfers on a High Performance Computing (HPC) cluster such as a Beowulf cluster. A peer-to-peer file tracker (PPFT) allows operating system, application and data files to be moved from a pre-loaded node to another node of the HPC cluster. A peer-to-peer (PTP) client is loaded into the nodes to facilitate PTP file transfers to reduce loading on networks, network switches and file servers to reduce the time needed to load the nodes with these files to increase overall efficiency of the multi-node computing system. The selection of the nodes participating in file transfers can be based on network topology, network utilization, job status and predicted network/computer utilization. This selection can be dynamic, changing during the file transfers as resource conditions change. The policies used to choose resources can be configured by an administrator. | 08-01-2013 |
20130198366 | DEPLOYING AN EXECUTABLE WITH HISTORICAL PERFORMANCE DATA - Techniques for incorporating performance data into an executable file for an application are described. Embodiments monitor performance of an application while the application is running. Additionally, historical execution characteristics of the application are determined based upon the monitored performance and one or more system characteristics of a node on which the application was executed on. Embodiments also incorporate the historical execution characteristics into executable file for the application, such that the historical execution characteristics can be used to manage subsequent executions of the application. | 08-01-2013 |
20130198371 | DEPLOYING AN EXECUTABLE WITH HISTORICAL PERFORMANCE DATA - Techniques for incorporating performance data into an executable file for an application are described. Embodiments monitor performance of an application while the application is running. Additionally, historical execution characteristics of the application are determined based upon the monitored performance and one or more system characteristics of a node on which the application was executed on. Embodiments also incorporate the historical execution characteristics into executable file for the application, such that the historical execution characteristics can be used to manage subsequent executions of the application. | 08-01-2013 |
20130198389 | DYNAMIC RESOURCE ADJUSTMENT FOR A DISTRIBUTED PROCESS ON A MULTI-NODE COMPUTER SYSTEM - A method dynamically adjusts the resources available to a processing unit of a distributed computer process executing on a multi-node computer system. The resources for the processing unit are adjusted based on the data other processing units handle or the execution path of code in an upstream or downstream processing unit in the distributed process or application. | 08-01-2013 |
20130198489 | PROCESSING ELEMENT MANAGEMENT IN A STREAMING DATA SYSTEM - Stream applications may inefficiently use the hardware resources that execute the processing elements of the data stream. For example, a compute node may host four processing elements and execute each using a CPU. However, other CPUs on the compute node may sit idle. To take advantage of these available hardware resources, a stream programmer may identify one or more processing elements that may be cloned. The cloned processing elements may be used to generate a different execution path that is parallel to the execution path that includes the original processing elements. Because the cloned processing elements contain the same operators as the original processing elements, the data stream that was previously flowing through only the original processing element may be split and sent through both the original and cloned processing elements. In this manner, the parallel execution path may use underutilized hardware resources to increase the throughput of the data stream. | 08-01-2013 |
20130254777 | DYNAMIC RUN TIME ALLOCATION OF DISTRIBUTED JOBS WITH APPLICATION SPECIFIC METRICS - A job optimizer dynamically changes the allocation of processing units on a multi-nodal computer system. A distributed application is organized as a set of connected processing units. The arrangement of the processing units is dynamically changed at run time to optimize system resources and interprocess communication. A collector collects application specific metrics determined by application plug-ins. A job optimizer analyzes the collected metrics and determines how to dynamically arrange the processing units within the jobs. The job optimizer may determine to combine multiple processing units into a job on a single node when there is an overutilization of an interprocess communication between processing units. Alternatively, the job optimizer may determine to split a job's processing units into multiple jobs on different nodes where one or more of the processing units are over utilizing the resources on the node. | 09-26-2013 |
20130262436 | OBTAINING PARTIAL RESULTS FROM A DATABASE QUERY - A database query governor determines when a query cannot be executed within a defined time threshold, but still executes the query or some variant of the query so partial results for the query may be returned to the user after executing for less than the defined time threshold, instead of an error message indicating the query was not executed at all. The partial results provide information to the user that may be sufficient for some applications, or may indicate to the user how the query could be modified to run in less time. | 10-03-2013 |
20130290394 | MONITORING STREAMS BUFFERING TO OPTIMIZE OPERATOR PROCESSING - Method, system and computer program product for performing an operation, including providing a plurality of processing elements comprising one or more operators, the operators configured to process streaming data tuples, establishing an operator graph of multiple operators, the operator graph defining at least one execution path in which a first operator is configured to receive data tuples from at least one upstream operator and transmit data tuples to at least one downstream operator, providing each operator a buffer configured to hold data tuples requiring processing by the respective operator, wherein the buffer is a first-in-first-out buffer, receiving a plurality of data tuples in a buffer associated with an operator, the data tuples comprising at least one attribute, selecting at least one data tuple from the first buffer, examining an attribute of the selected data tuples to identify a candidate tuple, and performing a second operation on the candidate tuple. | 10-31-2013 |
20130290489 | MONITORING STREAMS BUFFERING TO OPTIMIZE OPERATOR PROCESSING - Method, system and computer program product for performing an operation, including providing a plurality of processing elements comprising one or more operators, the operators configured to process streaming data tuples, establishing an operator graph of multiple operators, the operator graph defining at least one execution path in which a first operator is configured to receive data tuples from at least one upstream operator and transmit data tuples to at least one downstream operator, providing each operator a buffer configured to hold data tuples requiring processing by the respective operator, wherein the buffer is a first-in-first-out buffer, receiving a plurality of data tuples in a buffer associated with an operator, the data tuples comprising at least one attribute, selecting at least one data tuple from the first buffer, examining an attribute of the selected data tuples to identify a candidate tuple, and performing a second operation on the candidate tuple. | 10-31-2013 |
20130290966 | OPERATOR GRAPH CHANGES IN RESPONSE TO DYNAMIC CONNECTIONS IN STREAM COMPUTING APPLICATIONS - A stream computing application may permit one job to connect to a data stream of a different job. As more and more jobs dynamically connect to the data stream, the connections may have a negative impact on the performance of the job that generates the data stream. Accordingly, a variety of metrics and statistics (e.g., CPU utilization or tuple rate) may be monitored to determine if the dynamic connections are harming performance. If so, the stream computing system may be optimized to mitigate the effects of the dynamic connections. For example, particular operators may be unfused from a processing element and moved to a compute node that has available computing resources. Additionally, the stream computing application may clone the data stream in order to distribute the workload of transmitting the data stream to the connected jobs. | 10-31-2013 |
20130290969 | OPERATOR GRAPH CHANGES IN RESPONSE TO DYNAMIC CONNECTIONS IN STREAM COMPUTING APPLICATIONS - A stream computing application may permit one job to connect to a data stream of a different job. As more and more jobs dynamically connect to the data stream, the connections may have a negative impact on the performance of the job that generates the data stream. Accordingly, a variety of metrics and statistics (e.g., CPU utilization or tuple rate) may be monitored to determine if the dynamic connections are harming performance. If so, the stream computing system may be optimized to mitigate the effects of the dynamic connections. For example, particular operators may be unfused from a processing element and moved to a compute node that has available computing resources. Additionally, the stream computing application may clone the data stream in order to distribute the workload of transmitting the data stream to the connected jobs. | 10-31-2013 |
20130305032 | ANONYMIZATION OF DATA WITHIN A STREAMS ENVIRONMENT - Streams applications may decrypt encrypted data even though the decrypted data is not used by an operator. Operator properties are defined to permit decryption of data within the operator based on a number of criteria. By limiting the number of operators that decrypt encrypted data, the anonymous nature of the data is further preserved. Operator properties also indicate whether an operator should send encrypted or decrypted data to a downstream operator. | 11-14-2013 |
20130305034 | ANONYMIZATION OF DATA WITHIN A STREAMS ENVIRONMENT - Streams applications may decrypt encrypted data even though the decrypted data is not used by an operator. Operator properties are defined to permit decryption of data within the operator based on a number of criteria. By limiting the number of operators that decrypt encrypted data, the anonymous nature of the data is further preserved. Operator properties also indicate whether an operator should send encrypted or decrypted data to a downstream operator. | 11-14-2013 |
20130305225 | STREAMS DEBUGGING WITHIN A WINDOWING CONDITION - Method, system and computer program product for performing an operation, the operation including providing a plurality of processing elements comprising one or more operators, the operators configured to process streaming data tuples. The operation then establishes an operator graph of multiple operators, the operator graph defining at least one execution path in which a first operator of the plurality of operators is configured to receive data tuples from at least one upstream operator and transmit data tuples to at least one downstream operator. The operation then defines a breakpoint, the breakpoint comprising a condition, the condition based on attribute values of data tuples in a window of at least one operator, the window comprising a plurality of data tuples in an operator. The operation, upon detecting occurrence of the condition, triggers the breakpoint to halt processing by each of the plurality of operators in the operator graph. | 11-14-2013 |
20130305226 | Collecting Tracepoint Data - Collecting tracepoint data includes: maintaining, by a log handler, a list of a plurality of tracepoints and each tracepoint's parameters, the values of the parameters defining, for a tracepoint, information to be stored when an application encounters the tracepoint; encountering, by an application, a tracepoint; determining, responsive to the encounter by the log handler, whether the tracepoint is linked to a different tracepoint; and if the tracepoint is linked to a different tracepoint, determining, by the log handler, whether to modify values of the parameters for the different tracepoint and executing the different tracepoint. | 11-14-2013 |
20130305227 | STREAMS DEBUGGING WITHIN A WINDOWING CONDITION - Method product for performing an operation, the operation including providing a plurality of processing elements comprising one or more operators, the operators configured to process streaming data tuples. The operation then establishes an operator graph of multiple operators, the operator graph defining at least one execution path in which a first operator of the plurality of operators is configured to receive data tuples from at least one upstream operator and transmit data tuples to at least one downstream operator. The operation then defines a breakpoint, the breakpoint comprising a condition, the condition based on attribute values of data tuples in a window of at least one operator, the window comprising a plurality of data tuples in an operator. The operation, upon detecting occurrence of the condition, triggers the breakpoint to halt processing by each of the plurality of operators in the operator graph. | 11-14-2013 |
20140059556 | ENVIRONMENT BASED NODE SELECTION FOR WORK SCHEDULING IN A PARALLEL COMPUTING SYSTEM - A method, apparatus, and program product manage scheduling of a plurality of jobs in a parallel computing system of the type that includes a plurality of computing nodes and is disposed in a data center. The plurality of jobs are scheduled for execution on a group of computing nodes from the plurality of computing nodes based on the physical locations of the plurality of computing nodes in the data center. The group of computing nodes is further selected so as to distribute at least one of a heat load and an energy load within the data center. The plurality of jobs may be additionally scheduled based upon an estimated processing requirement for each job of the plurality of jobs. | 02-27-2014 |
20140089351 | HANDLING OUT-OF-SEQUENCE DATA IN A STREAMING ENVIRONMENT - Computer-implemented method, system, and computer program product for processing data in an out-of-order manner in a streams computing environment. A windowing condition is defined such that incoming data tuples are processed within a specified time or count of each other. Additionally, the windowing condition may be based on a specified attribute of the data tuples. If the tuples are not processed within the constraints specified by the windowing condition, the unprocessed tuples may be discarded, i.e., not processed, to optimize operator performance. | 03-27-2014 |
20140089352 | HANDLING OUT-OF-SEQUENCE DATA IN A STREAMING ENVIRONMENT - Computer-implemented method, system, and computer program product for processing data in an out-of-order manner in a streams computing environment. A windowing condition is defined such that incoming data tuples are processed within a specified time or count of each other. Additionally, the windowing condition may be based on a specified attribute of the data tuples. If the tuples are not processed within the constraints specified by the windowing condition, the unprocessed tuples may be discarded, i.e., not processed, to optimize operator performance. | 03-27-2014 |
20140089373 | DYNAMIC STREAM PROCESSING WITHIN AN OPERATOR GRAPH - A method and system for processing a stream of tuples in a stream-based application is disclosed. The method may include a first stream operator determining whether a requirement to modify processing of a first tuple at a second stream operator exists. The method may provide for associating an indication to modify processing of the first tuple at the second stream operator if the requirement exists. | 03-27-2014 |
20140089929 | DYNAMIC STREAM PROCESSING WITHIN AN OPERATOR GRAPH - A method and system for processing a stream of tuples in a stream-based application is disclosed. The method may include a first stream operator determining whether a requirement to modify processing of a first tuple at a second stream operator exists. The method may provide for associating an indication to modify processing of the first tuple at the second stream operator if the requirement exists. | 03-27-2014 |
20140095503 | COMPILE-TIME GROUPING OF TUPLES IN A STREAMING APPLICATION - A system and a method for initializing a streaming application are disclosed. The method may include initializing a streaming application for execution on one or more compute nodes which are adapted to execute one or more stream operators. The method may, during a compiling of code, identify whether a processing condition exists at a first stream operator of a plurality of stream operators. The method may add a grouping condition to a second stream operator of the plurality of stream operators if the processing condition exists. The method may provide for the second stream operator to group tuples for sending to the first stream operator. | 04-03-2014 |
20140095506 | COMPILE-TIME GROUPING OF TUPLES IN A STREAMING APPLICATION - A system and a method for initializing a streaming application are disclosed. The method may include initializing a streaming application for execution on one or more compute nodes which are adapted to execute one or more stream operators. The method may, during a compiling of code, identify whether a processing condition exists at a first stream operator of a plurality of stream operators. The method may add a grouping condition to a second stream operator of the plurality of stream operators if the processing condition exists. The method may provide for the second stream operator to group tuples for sending to the first stream operator. | 04-03-2014 |
20140122557 | RUNTIME GROUPING OF TUPLES IN A STREAMING APPLICATION - A system and method for modifying the processing within a streaming application are disclosed. The method may include identifying a grouping location at which it may be possible to group tuples during the runtime execution of a streaming application. In some embodiments, this may include identifying locations at which a runtime grouping condition may be added to one or more stream operators without adversely affecting the performance of a streaming application. The method may add a runtime grouping condition to a processing location within the plurality of stream operators of a streaming application, in some embodiments. | 05-01-2014 |
20140122559 | RUNTIME GROUPING OF TUPLES IN A STREAMING APPLICATION - A system and method for modifying the processing within a streaming application are disclosed. The method may include identifying a grouping location at which it may be possible to group tuples during the runtime execution of a streaming application. In some embodiments, this may include identifying locations at which a runtime grouping condition may be added to one or more stream operators without adversely affecting the performance of a streaming application. The method may add a runtime grouping condition to a processing location within the plurality of stream operators of a streaming application, in some embodiments. | 05-01-2014 |
20140136175 | IDENTIFYING AND ROUTING POISON TUPLES IN A STREAMING APPLICATION - A method for processing a stream of tuples may comprise receiving a stream of tuples to be processed by a plurality of processing elements operating on one or more computer processors. In addition, the method may include generating a model of performance for processing the stream of tuples at runtime, wherein one or more tuples from the stream of tuples potentially cause adverse performance. Further, the method may comprise predicting a parameter for a tuple from the stream of tuples, the parameter indicating a potential for adverse performance, the predicting including using the model. The method may also include modifying processing of the tuple if the parameter falls outside a threshold. | 05-15-2014 |
20140136176 | IDENTIFYING AND ROUTING POISON TUPLES IN A STREAMING APPLICATION - A method for processing a stream of tuples may comprise receiving a stream of tuples to be processed by a plurality of processing elements operating on one or more computer processors. In addition, the method may include generating a model of performance for processing the stream of tuples at runtime, wherein one or more tuples from the stream of tuples potentially cause adverse performance. Further, the method may comprise predicting a parameter for a tuple from the stream of tuples, the parameter indicating a potential for adverse performance, the predicting including using the model. The method may also include modifying processing of the tuple if the parameter falls outside a threshold. | 05-15-2014 |
20140136723 | STREAMS OPTIONAL EXECUTION PATHS DEPENDING UPON DATA RATES - Processing elements in a streaming application may contain one or more optional code modules—i.e., computer-executable code that is executed only if one or more conditions are met. In one embodiment, an optional code module is executed based on evaluating data flow rate between components in the streaming application. As an example, the stream computing application may monitor the incoming data rate between processing elements and select which optional code module to execute based on this rate. For example, if the data rate is high, the stream computing application may choose an optional code module that takes less time to execute. Alternatively, a high data rate may indicate that the incoming data is important; thus, the streaming application may choose an optional code module containing a more rigorous data processing algorithm, even if this algorithm takes more time to execute. | 05-15-2014 |
20140136724 | STREAMS OPTIONAL EXECUTION PATHS DEPENDING UPON DATA RATES - Processing elements in a streaming application may contain one or more optional code modules—i.e., computer-executable code that is executed only if one or more conditions are met. In one embodiment, an optional code module is executed based on evaluating data flow rate between components in the streaming application. As an example, the stream computing application may monitor the incoming data rate between processing elements and select which optional code module to execute based on this rate. For example, if the data rate is high, the stream computing application may choose an optional code module that takes less time to execute. Alternatively, a high data rate may indicate that the incoming data is important; thus, the streaming application may choose an optional code module containing a more rigorous data processing algorithm, even if this algorithm takes more time to execute. | 05-15-2014 |
20140164355 | TUPLE ROUTING IN A STREAMING APPLICATION - A system and method for modifying the processing within a streaming application are disclosed. The method may determine one or more parameters for a tuple at a first stream operator. The one or more parameters may represent a processing history of the tuple at the first stream operator. The method may associate the one or more parameters with the tuple metadata. A second stream operator may modify the processing of the tuple if the parameter falls outside a threshold. | 06-12-2014 |
20140164356 | TUPLE ROUTING IN A STREAMING APPLICATION - A system and method for modifying the processing within a streaming application are disclosed. The method may determine one or more parameters for a tuple at a first stream operator. The one or more parameters may represent a processing history of the tuple at the first stream operator. The method may associate the one or more parameters with the tuple metadata. A second stream operator may modify the processing of the tuple if the parameter falls outside a threshold. | 06-12-2014 |
20140164374 | STREAMING DATA PATTERN RECOGNITION AND PROCESSING - When processing data tuples, operators of a streaming application may identify certain tuples as being relevant. To determine relevant tuples, the operators may, for example, process the received tuples and determine if they meet certain thresholds. If so, the tuples are deemed relevant, but if not they are characterized as irrelevant. The streaming application may use a pattern detector to parse the relevant data tuples to identify a pattern, such as a shared trait between the tuples. Based on this commonality, the pattern detector may generate filtering criteria that may be used to process subsequently received tuples. In one embodiment, the filtering criteria identified by one operator is transmitted to a second operator to be used to process tuples received there. Thus, once one of the operators determines a pattern, the operator generates filtering criteria that another, related operator uses for filtering received tuples. | 06-12-2014 |
20140164434 | STREAMING DATA PATTERN RECOGNITION AND PROCESSING - When processing data tuples, operators of a streaming application may identify certain tuples as being relevant. To determine relevant tuples, the operators may, for example, process the received tuples and determine if they meet certain thresholds. If so, the tuples are deemed relevant, but if not they are characterized as irrelevant. The streaming application may use a pattern detector to parse the relevant data tuples to identify a pattern, such as a shared trait between the tuples. Based on this commonality, the pattern detector may generate filtering criteria that may be used to process subsequently received tuples. In one embodiment, the filtering criteria identified by one operator is transmitted to a second operator to be used to process tuples received there. Thus, once one of the operators determines a pattern, the operator generates filtering criteria that another, related operator uses for filtering received tuples. | 06-12-2014 |
20140164601 | MANAGEMENT OF STREAM OPERATORS WITH DYNAMIC CONNECTIONS - One embodiment is directed to a method for processing a stream of tuples in a stream-based application. A stream operator may receive a stream of tuples. A stream manager may determine whether a dynamic connection exists at a first stream operator. The dynamic connection may connect the first stream operator to a second stream operator. The stream manager may poll the first stream operator and the second stream operator for a presence of the dynamic connection. The stream manager may modify processing of one or more upstream stream operators in response to a change in use of the dynamic connection. | 06-12-2014 |
20140164628 | MANAGEMENT OF STREAM OPERATORS WITH DYNAMIC CONNECTIONS - One embodiment is directed to a method for processing a stream of tuples in a stream-based application. A stream operator may receive a stream of tuples. A stream manager may determine whether a dynamic connection exists at a first stream operator. The dynamic connection may connect the first stream operator to a second stream operator. The stream manager may poll the first stream operator and the second stream operator for a presence of the dynamic connection. The stream manager may modify processing of one or more upstream stream operators in response to a change in use of the dynamic connection. | 06-12-2014 |
20140215165 | MEMORY MANAGEMENT IN A STREAMING APPLICATION - One embodiment is directed to a method for processing a stream of tuples. The method may include receiving a stream of tuples to be processed by a plurality of processing elements operating on one or more computer processors. Each of the processing elements has an associated memory space. In addition, the method may include monitoring the plurality of processing elements. The monitoring may include identifying a first performance metric for a first processing element. The method may include modifying the first processing element based on the first performance metric. The modifying of the first processing element may include employing memory management of the associated memory space. | 07-31-2014 |
20140215184 | MEMORY MANAGEMENT IN A STREAMING APPLICATION - One embodiment is directed to a method for processing a stream of tuples. The method may include receiving a stream of tuples to be processed by a plurality of processing elements operating on one or more computer processors. Each of the processing elements has an associated memory space. In addition, the method may include monitoring the plurality of processing elements. The monitoring may include identifying a first performance metric for a first processing element. The method may include modifying the first processing element based on the first performance metric. The modifying of the first processing element may include employing memory management of the associated memory space. | 07-31-2014 |
20140236920 | STREAMING DELAY PATTERNS IN A STREAMING ENVIRONMENT - The method and system receive streaming data to be processed by a plurality of processing elements comprising one or more stream operators. One embodiment is directed to a method and a system for managing processing in a streaming application. A stream operator may select a delay pattern. The stream operator may compare one or more performance factors from the delay pattern to one or more optimal performance factors. The stream operator may delay the stream of tuples using the delay pattern if the performance factors are determined by the optimal performance factors. | 08-21-2014 |
20140237134 | STREAMING DELAY PATTERNS IN A STREAMING ENVIRONMENT - The method and system receive streaming data to be processed by a plurality of processing elements comprising one or more stream operators. One embodiment is directed to a method and a system for managing processing in a streaming application. A stream operator may select a delay pattern. The stream operator may compare one or more performance factors from the delay pattern to one or more optimal performance factors. The stream operator may delay the stream of tuples using the delay pattern if the performance factors are determined by the optimal performance factors. | 08-21-2014 |
20140250103 | OBTAINING PARTIAL RESULTS FROM A DATABASE QUERY - A database query governor determines when a query cannot be executed within a defined time threshold, but still executes the query or some variant of the query so partial results for the query may be returned to the user after executing for less than the defined time threshold, instead of an error message indicating the query was not executed at all. The partial results provide information to the user that may be sufficient for some applications, or may indicate to the user how the query could be modified to run in less time. | 09-04-2014 |
20140258290 | PROCESSING CONTROL IN A STREAMING APPLICATION - A method, system, and computer program product for processing a stream of tuples are disclosed. The method, system, and computer program product may include receiving a stream of tuples to be processed by a plurality of processing elements. Each tuple may have an associated processing history. The stream of tuples may be segmented into a plurality of partitions, each representing a subset of the stream of tuples. The method, system, and computer program product may include estimating the contribution each partition will have on a particular processing result and processing a partition if it substantially contributes to the particular processing result. | 09-11-2014 |
20140258291 | PROCESSING CONTROL IN A STREAMING APPLICATION - A method, system, and computer program product for processing a stream of tuples are disclosed. The method, system, and computer program product may include receiving a stream of tuples to be processed by a plurality of processing elements. Each tuple may have an associated processing history. The stream of tuples may be segmented into a plurality of partitions, each representing a subset of the stream of tuples. The method, system, and computer program product may include estimating the contribution each partition will have on a particular processing result and processing a partition if it substantially contributes to the particular processing result. | 09-11-2014 |
20140278337 | SELECTING AN OPERATOR GRAPH CONFIGURATION FOR A STREAM-BASED COMPUTING APPLICATION - First and second simulated processing of a stream-based computing application using respective first and second simulation conditions may be performed. The first and second simulation conditions may specify first and second operator graph configurations. Each simulated processing may include inputting a stream of test tuples to the stream-based computing application, which may operate on one or more compute nodes. Each compute node may have one or more computer processors and a memory to store one or more processing elements. Each simulated processing may be monitored to determine one or more performance metrics. The first and second simulated processings may be sorted based on a first performance metric to identify a simulated processing having a first rank. An operator graph configuration associated with the simulated processing having the first rank may be selected if the first performance metric for the simulated processing having the first rank is within a processing constraint. | 09-18-2014 |
20140279965 | COMPRESSING TUPLES IN A STREAMING APPLICATION - A method, system, and computer program product to process data in a streaming application are disclosed. The method, system, and computer program product may include receiving a stream of tuples to be processed by a plurality of processing elements operating on a plurality of compute nodes. The method, system, and computer program product may determine whether a first processing element has additional processing capacity. In some embodiments, the method, system, and computer program product determine whether a second processing element, which receives its input from the first processing element, also has additional processing capacity. The method, system, and computer program product may employ compression at the first processing element if one of the first and the second processing element has additional processing capacity. | 09-18-2014 |
20140279968 | COMPRESSING TUPLES IN A STREAMING APPLICATION - A method, system, and computer program product to process data in a streaming application are disclosed. The method, system, and computer program product may include receiving a stream of tuples to be processed by a plurality of processing elements operating on a plurality of compute nodes. The method, system, and computer program product may determine whether a first processing element has additional processing capacity. In some embodiments, the method, system, and computer program product determine whether a second processing element, which receives its input from the first processing element, also has additional processing capacity. The method, system, and computer program product may employ compression at the first processing element if one of the first and the second processing element has additional processing capacity. | 09-18-2014 |
20140280128 | ENDING TUPLE PROCESSING IN A STREAM-BASED COMPUTING APPLICATION - A method includes receiving streaming data to be processed by a plurality of processing elements comprising one or more stream operators. Time metadata may be added to a parent tuple at a first stream operator. A first time metric may be determined for a first child tuple of the parent tuple at a second stream operator. The first time metric may be determined, at least in part, from the time metadata. The second stream operator may receive the first child tuple from the first stream operator. The method may include transmitting a second child tuple of the parent tuple from the second stream operator to a third stream operator if the time metric is inside a time limit. In addition, the method may include ending processing of the first child tuple if the time metric is outside of the time limit. | 09-18-2014 |
20140280895 | EVALUATING A STREAM-BASED COMPUTING APPLICATION - A method for evaluating a stream-based computing application includes specifying a simulation condition. In addition, a stream of test tuples may be input to the stream-based computing application. The stream-based computing application may operate on one or more compute nodes. Each compute node may have one or more computer processors and a memory to store one or more processing elements. The method may also include simulating processing of the stream of test tuples by the processing elements using the simulation condition. Further, the method may include monitoring to determine one or more performance metrics for an inter-stream operator communication path. | 09-18-2014 |
20140289186 | MANAGING ATTRIBUTES IN STREM PROCESSING USING A CACHE - A method and system for managing attributes in a streaming application is disclosed. The system may contain a receiving stream operator that is communicatively coupled with a stream manager. The receiving stream operator may have a capability of storing a selected attribute and creating one or more unique identifiers. The system may contain a cache communicatively coupled with one or more stream operators. The cache may have a capability of storing the selected attributes. The system may also have a retrieving stream operator communicatively coupled with the stream manager. The retrieving stream operator may have a capability of using the unique identifier to access the selected attribute. | 09-25-2014 |
20140289240 | MANAGING ATTRIBUTES IN STREAM PROCESSING - A method and system for managing attributes in a streaming application is disclosed. The system may have a stream manager communicatively coupled with processing elements for tracking a stream of tuples. The system may also have a first stream operator communicatively coupled with the stream manager and capable of receiving the stream of tuples, wherein the first stream operator selects the selected attribute of the first tuple and assigns a first identifier to the selected attribute. The system may also have a second stream operator communicatively coupled with the stream manager and capable of receiving the stream of tuples, and capable of replacing the selected attribute in the second tuple with a second identifier provided by the first stream operator. The system may also have an identifier table communicatively coupled with the stream manager and the first and second stream operator, wherein the identifier table includes identifiers for selected tuples. | 09-25-2014 |
20140317148 | RECONFIGURING AN OPERATOR GRAPH BASED ON ATTRIBUTE USAGE - A first processing element may be initially configured to transmit a first output stream to a second processing element. The second processing element may be initially configured to transmit a second output stream to a third processing element. The tuples of the first and second output streams may have the first and second attributes. It may be determined whether the first attribute is to be first processed at the second processing element (first condition) and whether the second attribute is to be first processed at the third processing element (second condition). When the first and second conditions are met, the first processing element may be reconfigured to transmit a third output stream to the second processing element and a fourth output stream to the third processing element. The third output stream may have only the first attribute. The fourth output stream may have only the second attribute. | 10-23-2014 |
20140317150 | EXITING WINDOWING EARLY FOR STREAM COMPUTING - Two or more tuples to be processed by a processing element operating on one or more computer processors may be received by the processing element. The processing element may have a windowing operator performing a windowing operation to determine a first value at the conclusion of a windowing condition. It may be determined from one or more tuples received within the windowing condition whether a condition to end the windowing operation before the windowing condition concludes is met. In addition, the windowing operation may be ended before the windowing condition concludes when the condition to end the windowing operation is met. | 10-23-2014 |
20140317151 | EXITING WINDOWING EARLY FOR STREAM COMPUTING - Two or more tuples to be processed by a processing element operating on one or more computer processors may be received by the processing element. The processing element may have a windowing operator performing a windowing operation to determine a first value at the conclusion of a windowing condition. It may be determined from one or more tuples received within the windowing condition whether a condition to end the windowing operation before the windowing condition concludes is met. In addition, the windowing operation may be ended before the windowing condition concludes when the condition to end the windowing operation is met. | 10-23-2014 |
20140317304 | RUNTIME TUPLE ATTRIBUTE COMPRESSION - A method, system, and computer program product for initializing a stream computing application are disclosed. The method may include receiving a plurality of tuples to be processed by one or more processing elements operating on one or more computer processors. Each processing element may have one or more stream operators. The method may also include determining a first attribute to be processed at a first stream operator that is configured to transmit a tuple having the first attribute along an execution path including at least one intervening stream operator to a second stream operator. The method may include compressing the first attribute when the first attribute is to be next processed by the second stream operator. | 10-23-2014 |
20140317305 | COMPILE-TIME TUPLE ATTRIBUTE COMPRESSION - A method, system, and computer program product for initializing a stream computing application are disclosed. The method may include, during a compiling of code, determining whether an attribute of a tuple to be processed at a first stream operator is to be next processed at a second stream operator. The first stream operator may be configured to transmit the tuple along an execution path to the second stream operator. The execution path includes one or more intervening stream operators between the first and second stream operators. The method may invoke a compression condition when the first attribute of the tuple to be processed at the first stream operator is to be next processed at the second stream operator. | 10-23-2014 |
20140365612 | MONITORING SIMILAR DATA IN STREAM COMPUTING - A method, system, and computer program product for monitoring similar data in stream computing are disclosed. The method may include, monitoring at least one input stream of tuples to be processed by an application. The application may comprise one or more processing elements operating on one or more computer processors and each tuple is an instance of data. The method may also include, identifying a first tuple in the input stream and the first tuple is a first instance of first data. Also, the method may include, identifying a second tuple in the input stream and the second tuple is a second instance of first data. Furthermore, the method may include, determining that the second tuple satisfies criteria for superseding the first tuple and eliminating the first tuple from the application. | 12-11-2014 |
20140365614 | MONITORING SIMILAR DATA IN STREAM COMPUTING - A method, system, and computer program product for monitoring similar data in stream computing are disclosed. The method may include, monitoring at least one input stream of tuples to be processed by an application. The application may comprise one or more processing elements operating on one or more computer processors and each tuple is an instance of data. The method may also include, identifying a first tuple in the input stream and the first tuple is a first instance of first data. Also, the method may include, identifying a second tuple in the input stream and the second tuple is a second instance of first data. Furthermore, the method may include, determining that the second tuple satisfies criteria for superseding the first tuple and eliminating the first tuple from the application. | 12-11-2014 |
20140372431 | GENERATING DIFFERENCES FOR TUPLE ATTRIBUTES - A sequence of tuples, each having one or more attributes, is received at one of one or more processing elements operating on one or more processors. Each processing element may have one or more stream operators. A first stream operator may be identified as one that only processes an instance of a first attribute in a currently received tuple when a difference between an instance of the first attribute in a previously received tuple and the instance of the first attribute in the currently received tuple is outside of a difference threshold. A second stream operator may generate a difference attribute from a first instance of the first attribute in a first one of the received tuples and a second instance of the first attribute in a second one of the received tuples. The difference attribute may be transmitted from the second stream operator to the first stream operator. | 12-18-2014 |
20140373019 | GENERATING DIFFERENCES FOR TUPLE ATTRIBUTES - A sequence of tuples, each having one or more attributes, is received at one of one or more processing elements operating on one or more processors. Each processing element may have one or more stream operators. A first stream operator may be identified as one that only processes an instance of a first attribute in a currently received tuple when a difference between an instance of the first attribute in a previously received tuple and the instance of the first attribute in the currently received tuple is outside of a difference threshold. A second stream operator may generate a difference attribute from a first instance of the first attribute in a first one of the received tuples and a second instance of the first attribute in a second one of the received tuples. The difference attribute may be transmitted from the second stream operator to the first stream operator. | 12-18-2014 |
20140379711 | MANAGING PASSTHRU CONNECTIONS ON AN OPERATOR GRAPH - Embodiments of the disclosure provide a method, system, and computer program product for processing data such as a stream of tuples. Each tuple can contain one or more attributes. The method can include processing the attributes of the stream of tuples using stream operators operating on one or more computer processors and corresponding to one or more processing elements. The method can also include detecting an indicative element from a plurality of stream operators. The method can also include transmitting, in response to detecting the indicative element, a passthru command to a processing element corresponding to the indicative element. The method can also include altering, in response to receiving the passthru command at the processing element, a portion of attribute processing for the indicative element. | 12-25-2014 |
20150074108 | ENDING TUPLE PROCESSING IN A STREAM-BASED COMPUTING APPLICATION - A method includes receiving streaming data to be processed by a plurality of processing elements comprising one or more stream operators. Time metadata may be added to a parent tuple at a first stream operator. A first time metric may be determined for a first child tuple of the parent tuple at a second stream operator. The first time metric may be determined, at least in part, from the time metadata. The second stream operator may receive the first child tuple from the first stream operator. The method may include transmitting a second child tuple of the parent tuple from the second stream operator to a third stream operator if the time metric is inside a time limit. In addition, the method may include ending processing of the first child tuple if the time metric is outside of the time limit. | 03-12-2015 |
20150081693 | MANAGING DATA PATHS IN AN OPERATOR GRAPH - Embodiments of the disclosure provide a method and, system for processing data such as a stream of tuples. The method can include receiving the stream of tuples to be processed by a plurality of stream operators operating on one or more computer processors. The method can include creating an overflow path that includes at least one stream operator that performs processing duplicative to at least one stream operator from the plurality of stream operators. The method can include monitoring a stream operator for a triggering condition. The method can include identifying a tuple from the stream of tuples to process on the overflow path. The method can include processing, on the overflow path, the identified tuple from the stream of tuples in response to the presence of the triggering condition. | 03-19-2015 |
20150081707 | MANAGING A GROUPING WINDOW ON AN OPERATOR GRAPH - Embodiments of the disclosure provide a method, system, and computer program product for managing a windowing operation. The method can include determining a sentinel value that defines a start of a grouping window for a stream of tuples and a terminating sentinel value that defines the end of the grouping window based upon an attribute contained in the stream of tuples. The stream of tuples can be monitored for the sentinel value and the terminating sentinel value by a stream operator. The stream operator can initiate a windowing operation that defines the start of the grouping window in response to a presence of the sentinel value and terminate the windowing operation in response to a presence of the terminating sentinel value. | 03-19-2015 |
20150081708 | MANAGING A GROUPING WINDOW ON AN OPERATOR GRAPH - Embodiments of the disclosure provide a method, system, and computer program product for managing a windowing operation. The method can include determining a sentinel value that defines a start of a grouping window for a stream of tuples and a terminating sentinel value that defines the end of the grouping window based upon an attribute contained in the stream of tuples. The stream of tuples can be monitored for the sentinel value and the terminating sentinel value by a stream operator. The stream operator can initiate a windowing operation that defines the start of the grouping window in response to a presence of the sentinel value and terminate the windowing operation in response to a presence of the terminating sentinel value. | 03-19-2015 |
20150081879 | MANAGING DATA PATHS IN AN OPERATOR GRAPH - Embodiments of the disclosure provide a method and, system for processing data such as a stream of tuples. The method can include receiving the stream of tuples to be processed by a plurality of stream operators operating on one or more computer processors. The method can include creating an overflow path that includes at least one stream operator that performs processing duplicative to at least one stream operator from the plurality of stream operators. The method can include monitoring a stream operator for a triggering condition. The method can include identifying a tuple from the stream of tuples to process on the overflow path. The method can include processing, on the overflow path, the identified tuple from the stream of tuples in response to the presence of the triggering condition. | 03-19-2015 |
20150088887 | MANAGING MULTIPLE WINDOWS ON AN OPERATOR GRAPH - Embodiments of the disclosure provide a method, system, and computer program product for managing a windowing operation. The method for grouping processing of a stream of tuples with each tuple containing one or more attributes can include receiving the stream of tuples to be processed by a plurality of processing elements operating on one or more computer processors. The method can also include processing, with a first processing method, a group of tuples from the stream of tuples into a grouping window. The method can also include processing, with a second processing method, a subgroup of tuples from the group of tuples into a subgrouping window. The second processing method can include identifying a sub-membership condition. | 03-26-2015 |
20150088889 | MANAGING MULTIPLE WINDOWS ON AN OPERATOR GRAPH - Embodiments of the disclosure provide a method, system, and computer program product for managing a windowing operation. The method for grouping processing of a stream of tuples with each tuple containing one or more attributes can include receiving the stream of tuples to be processed by a plurality of processing elements operating on one or more computer processors. The method can also include processing, with a first processing method, a group of tuples from the stream of tuples into a grouping window. The method can also include processing, with a second processing method, a subgroup of tuples from the group of tuples into a subgrouping window. The second processing method can include identifying a sub-membership condition. | 03-26-2015 |