Patent application number | Description | Published |
20080243865 | Maintaining global state of distributed transaction managed by an external transaction manager for clustered database systems - In accordance with an embodiment of the present invention, a transaction tracking mechanism is provided by a database server cluster to keep track of a global state of a distributed transaction. The global state of the distributed transaction comprises one or more statuses that are associated with one or more transaction branches that are part of the distributed transaction. The global state may be associated with a global lock. Through using the global state in association with the global lock, problems such as partial commits, data inconsistency, access contentions and deadlocks may be avoided when the database server cluster processes the distributed transaction. | 10-02-2008 |
20080281846 | High performant row-level data manipulation using a data layer interface - Performing an operation directly on data stored in a database table provides for avoiding generation and processing of SQL statements. A Data Layer interface, comprising a set of APIs, allows internal database server components to call directly to the Data Layer for making fast row changes to a table. According to an embodiment, before making any row change to a table through this Data Layer interface, a shared state data structure is created to hold the metadata about the table. A single shared state can be used for all DML changes to a particular table regardless of what columns are updated or inserted during each DML operation. When a process needs to change a row, the process uses a mutable state based on the shared state of the table. | 11-13-2008 |
20090024578 | Delta Operations On A Large Object In A Database - A computer modifies data inside a database large object (LOB) of unknown structure without modifying other data in the remainder of the LOB. Insertion of new data at a specified location in the LOB does not require movement of existing data in the LOB. Instead, the computer is programmed to insert new data at a physical end of the LOB, and modify metadata based on the specified location. Similarly, deletion of existing data from a specified location in the LOB is performed without movement of other data in the LOB, by updating the metadata. The computer uses the metadata when reading from the LOB, so that the new data is automatically read whenever the specified location is accessed. The computer may optionally output a handle that is static, relative to other insertions and deletions, to identify specific data within the LOB, for use in building indexes on the LOB. | 01-22-2009 |
20090030956 | Proactive space allocation in a database system - A method and apparatus for proactively allocating space to a logical layer of a database is provided. Allocation of space to a logical layer of a database is an expensive operation that should be minimized. The allocation of space includes discovering free space in all the higher layers in the logical storage hierarchy. It also includes cross-domain system calls and cleansing disk-write operations. In prior approaches, the allocation processes were triggered on-demand, only a certain logical layer was discovered to be at full capacity when an insert-row operation was attempted. In one embodiment of the invention, space needs are statistically predicted based on prior rates of space consumption. The database server pre-allocates space as needed, based on the statistical predictions. Rates of consumption are examined periodically. Space is pre-allocated before any logical layer reaches full capacity by a combination of proactive background processes; foreground-triggered, background processes; and classic foreground allocation. | 01-29-2009 |
20090037366 | ONLINE INDEX BUILDS AND REBUILDS WITHOUT BLOCKING LOCKS - Techniques are provided for processing a CREATE INDEX statement that avoids one or more locks on the underlying table. The processing of such a statement may be divided into a build phase and a cooperation phase. During the build phase, a journal table is created and published. A snapshot of the table is obtained after the last DML transaction (that was pending at the time the journal table was published) commits. The index is built based on the state of the table as of the snapshot. Any changes to the table while the index is built are recorded in the journal table. During the cooperation phase, the index is partially published. A subsequent DML statement determines whether any changes in the journal table affect the same data as the subsequent DML statement. If so, then those changes are applied to the index followed by the subsequent DML statement. | 02-05-2009 |
20090037498 | IN-MEMORY SPACE MANAGEMENT FOR DATABASE SYSTEMS - A framework for in-memory space management for content management database systems is provided. A per-instance in-memory dispenser is partitioned. An incoming transaction takes a latch on a partition and obtains sufficient block usage to perform and complete the transaction. Generating redo information is decoupled from transaction level processing and, instead, is performed when block requests are loaded into the in-memory dispenser or synced therefrom to a per-instance on-disk structure. | 02-05-2009 |
20090037499 | METHOD AND MECHANISM FOR IMPLEMENTING DYNAMIC SPACE MANAGEMENT FOR LARGE OBJECTS - Disclosed is a system and method for implementing space management for large objects stored in the computing system. According to some approaches, storage of large objects are managed by dynamically creating contiguous chunks of storage space of varying lengths. The length of each chunk may vary depending upon object size being stored, fragmentation of the storage space, available free space, and/or expected length of the object. | 02-05-2009 |
20090106281 | ON-LINE TRANSACTION PROCESSING (OLTP) COMPRESSION AND RE-COMPRESSION OF DATABASE DATA - A computer is programmed to compress data of a database in response to database modification language (DML) statements generated by on-line transaction processing (OLTP) systems. In several embodiments, data that is initially added to a database block is left uncompressed until a predetermined condition is satisfied, which happens infrequently (relative to OLTP transactions on the block). When satisfied, the computer automatically compresses all uncompressed data in the block, which increases the amount of unused space in the block. New data is thereafter added uncompressed to the partially compressed block, until satisfaction of a predetermined condition whereby the partially compressed block is again compressed, i.e. re-compressed. Adding of new data to a partially compressed block and its compression are repeated unless another predetermined condition is met, in response to which the block is not further re-compressed, thereby to recognize a limit on the benefit from compression. | 04-23-2009 |
20090164525 | Online Recovery of User Tables Using Flashback Table - A method and apparatus for updating databases are disclosed. In response to a flashback request, a database object is restored to a state as of a specified flashback time. The changes introduced into the database object are reversed in a sequence starting with a change introduced into the database object immediately before issuance of the flashback request and ending with a change introduced into the database object immediately after the specified flashback time. | 06-25-2009 |
20090292679 | CASCADING INDEX COMPRESSION - Techniques for compressing branch nodes in an index are provided. The branch nodes may be part of a main index of a multi-level index that also includes one or more journal indexes. A Bloom filter may be generated and associated with, e.g., a branch node in the main index. The Bloom filter is used to determine whether, without accessing any leaf blocks, a particular key value exists, e.g., in leaf blocks associated with the branch node. | 11-26-2009 |
20090292947 | CASCADING INDEX COMPRESSION - Techniques for maintaining a cascading index are provided. In one approach, one or more branch node compression techniques are applied to the main index of a cascading index. In an approach, a Bloom filter is generated and associated with, e.g., a branch node in the main index. The Bloom filter is used to determine whether, without accessing any leaf blocks, a particular key value exists, e.g., in leaf blocks associated with the branch node. In an approach, a new redo record is generated in response to a merge operation between two levels of the cascading index. The new redo record comprises (a) one or more addresses of blocks that are affected by the merge operation, (b) data is that being “pushed down” to a lower level of the cascading index, and (c) one or more addresses of blocks that are written to disk storage as a result of the merge operation. | 11-26-2009 |
20100030781 | METHOD AND APPARATUS FOR AUTOMATICALLY CLASSIFYING DATA - One embodiment of the present invention provides a system for automatically classifying data in a database. During operation, the system receives and executes a database operation. Next, the system automatically determines if any data was modified as a result of executing the database operation. If so, for each data item that was modified, the system automatically determines if the data item is associated with a classification-rule. If so, the system automatically reclassifies the data item according to the classification-rule. If not, the system leaves a classification of the data item unchanged. | 02-04-2010 |
20100036803 | ADAPTIVE FILTER INDEX FOR DETERMINING QUERIES AFFECTED BY A DML OPERATION - Techniques are disclosed for creating and using a filter index in order to identify registered queries whose result sets are likely to have been changed by changes made to tables. The filter index entries are based on filter conditions. The filter conditions are created based on predicates contained in the registered queries. The filter conditions may include exclusive predicates and join predicates. Join predicates that join a table T | 02-11-2010 |
20100036831 | GENERATING CONTINUOUS QUERY NOTIFICATIONS - Techniques are described to allow a query to be registered as a persistent stored entity within the database, and to generate notifications as and when the query result changes continuously as long as the query continues to be registered with the database. According to one aspect, for a table referenced in a query, a filter condition is generated based, at least in part, on a predicate of the query. Then, the database server determines whether the filter condition is satisfied by either a before image of a row, or an after image of the row, that was modified by a transaction. If the filter condition is satisfied by either the before image or the after image, then the query is added to a first set of queries whose result sets may have been affected by the transaction. From among the first set of queries, a second set of queries that have result sets that were actually affected by the transaction is determined. Notifications are then sent based on the second set of queries. | 02-11-2010 |
20100278446 | STRUCTURE OF HIERARCHICAL COMPRESSED DATA STRUCTURE FOR TABULAR DATA - A highly flexible and extensible structure is provided for physically storing tabular data. The structure, is referred to as a compression unit, and may be used to physically store tabular data that logically resides in any type of table-like structure. According to one embodiment, compression units are recursive. Thus, a compression unit may have a “parent” compression unit to which it belongs, and may have one or more “child” compression units that belong to it. In one embodiment, compression units include metadata that indicates how the tabular data is stored within them. The metadata for a compression unit may indicate, for example, whether the data within the compression unit is stored in row-major or column major-format (or some combination thereof), the order of the columns within the compression unit (which may differ from the logical order of the columns dictated by the definition of their logical container), a compression technique for the compression unit, the child compression units (if any), etc. | 11-04-2010 |
20100281004 | STORING COMPRESSION UNITS IN RELATIONAL TABLES - A database server stores compressed units in data blocks of a database. A table (or data from a plurality of rows thereof) is first compressed into a “compression unit” using any of a wide variety of compression techniques. The compression unit is then stored in one or more data block rows across one or more data blocks. As a result, a single data block row may comprise compressed data for a plurality of table rows, as encoded within the compression unit. Storage of compression units in data blocks maintains compatibility with existing data block-based databases, thus allowing the use of compression units in preexisting databases without modification to the underlying format of the database. The compression units may, for example, co-exist with uncompressed tables. Various techniques allow a database server to optimize access to data in the compression unit, so that the compression is virtually transparent to the user. | 11-04-2010 |
20100281079 | COMPRESSION ANALYZER - Techniques are described herein for automatically selecting the compression techniques to be used on tabular data. A compression analyzer gives users high-level control over the selection process without requiring the user to know details about the specific compression techniques that are available to the compression analyzer. Users are able to specify, for a given set of data, a “balance point” along the spectrum between “maximum performance” and “maximum compression”. The point thus selected is used by the compression analyzer in a variety of ways. For example, in one embodiment, the compression analyzer uses the user-specified balance point to determine which of the available compression techniques qualify as “candidate techniques” for the given set of data. The compression analyzer selects the compression technique to use on a set of data by actually testing the candidate compression techniques against samples from the set of data. After testing the candidate compression techniques against the samples, the resulting compression ratios are compared. The compression technique to use on the set of data is then selected based, in part, on the compression ratios achieved during the compression tests performed on the sample data. | 11-04-2010 |
20110029569 | DDL AND DML SUPPORT FOR HYBRID COLUMNAR COMPRESSED TABLES - Techniques for storing and manipulating tabular data are provided. According to one embodiment, a user may control whether tabular data is stored in row-level or column-major format. Furthermore, the user may control the level of data compression to achieve an optimal balance between query performance and compression ratios. Tabular data from within the same table may be stored in both column-major and row-major format and compressed at different levels. In addition, tabular data can migrate between column-major format and row-major format in response to various events. For example, in response to a request to update or lock a row stored in column-major format, the row may be migrated and subsequently stored into row-major format. In one embodiment, table partitions are used to enhance data compression techniques. For example, compression tests are performed on a representative table partition, and a compression map is generated and applied to other table partitions. | 02-03-2011 |
20110302205 | METHOD AND MECHANISM FOR DETERMINING LIFE INTERVALS FOR A KEY IN AN INDEX - Disclosed is a mechanism, system, interface, and process to determine one or more life intervals when given a key. In addition, one or more life intervals can be determined for each key within a range of keys. This type of process is useful for auditing purposes or if it is desire to implement a query upon past versions when a key or range of keys is specified for the data for which past versions is desired. This facilitates algorithms, mechanisms, and processes to implement a query upon past versions. | 12-08-2011 |
20120117038 | LAZY OPERATIONS ON HIERARCHICAL COMPRESSED DATA STRUCTURE FOR TABULAR DATA - A highly flexible and extensible structure is provided for physically storing tabular data. The structure, referred to as a compression unit, may be used to physically store tabular data that logically resides in any type of table-like structure. Techniques are employed to avoid changing tabular data within existing compression units. Deleting tabular data within compression units is avoided by merely tracking deletion requests, without actually deleting the data. Inserting new tabular data into existing compression units is avoided by storing the new data external to the compression units. If the number of deletions exceeds a threshold, and/or the number of new inserts exceeds a threshold, new compression units may be generated. When new compression units are generated, the previously-existing compression units may be discarded to reclaim storage, or retained to allow reconstruction of prior states of the tabular data. | 05-10-2012 |
20120143833 | STRUCTURE OF HIERARCHICAL COMPRESSED DATA STRUCTURE FOR TABULAR DATA - A highly flexible and extensible structure is provided for physically storing tabular data. The structure, referred to as a compression unit, may be used to store tabular data that logically resides in any type of table-like structure. According to one embodiment, compression units are recursive. Thus, a compression unit may have a “parent” compression unit to which it belongs, and may have one or more “child” compression units that belong to it. In one embodiment, compression units include metadata that indicates how the tabular data is stored within them. The metadata for a compression unit may indicate, for example, whether the data is stored in row-major or column major-format the order of the columns within the compression unit (which may differ from the logical order of the columns dictated by the definition of their logical container), a compression technique for the compression unit, the child compression units (if any), etc. | 06-07-2012 |
20120296883 | Techniques For Automatic Data Placement With Compression And Columnar Storage - For automatic data placement of database data, a plurality of access-tracking data is maintained. The plurality of access-tracking data respectively corresponds to a plurality of data rows that are managed by a database server. While the database server is executing normally, it is automatically determined whether a data row, which is stored in first one or more data blocks, has been recently accessed based on the access-tracking data that corresponds to that data row. After determining that the data row has been recently accessed, the data row is automatically moved from the first one or more data blocks to one or more hot data blocks that are designated for storing those data rows, from the plurality of data rows, that have been recently accessed. | 11-22-2012 |
20130036101 | Compression Analyzer - Techniques are described herein for automatically selecting the compression techniques to be used on tabular data. A compression analyzer gives users high-level control over the selection process without requiring the user to know details about the specific compression techniques that are available to the compression analyzer. Users are able to specify, for a given set of data, a “balance point” along the spectrum between “maximum performance” and “maximum compression”. The point thus selected is used by the compression analyzer in a variety of ways. For example, in one embodiment, the compression analyzer uses the user-specified balance point to determine which of the available compression techniques qualify as “candidate techniques” for the given set of data. The compression analyzer selects the compression technique to use on a set of data by actually testing the candidate compression techniques against samples from the set of data. After testing the candidate compression techniques against the samples, the resulting compression ratios are compared. The compression technique to use on the set of data is then selected based, in part, on the compression ratios achieved during the compression tests performed on the sample data. | 02-07-2013 |
20130144842 | FAILOVER AND RESUME WHEN USING ORDERED SEQUENCES IN A MULTI-INSTANCE DATABASE ENVIRONMENT - An approach is disclosed for implementing failover and resume when using ordered sequences in a multi-instance database environment. The approach commences by instantiating a first database instance initially to serve as an active instance, then instantiating a second database instance to serve as an instance of one or more passive instances. The active database establishes mastership over a sequence and then processes requests for the ‘next’ symbol by accessing a shared sequence cache only after accessing a first instance semaphore. The active instance and the passive instance perform a protocol such that upon passive database detection of a failure of the active database, one of the passive database instances takes over mastership of the sequence cache, and then proceeds to satisfy sequence value requests. The particular order is observed in spite of the failure. | 06-06-2013 |
20130145103 | REDUCING SEQUENCE CACHE LATCH CONTENTION IN A DATABASE SYSTEM - In a database system having a plurality of concurrently executing session processes, the method commences by establishing a master list of sequences, the master list comprising a plurality of sequence objects which in turn define a sequence of values used for numbering and other identification within the database system. To reduce sequence cache latch access contention, multiple tiers of latches are provided. Methods of the system provide a first tier having a first tier “global” latch to serialize access to the master list such that at any point in time, only one of the concurrently executing session processes is granted access to the master list, from which master list are allocated sequences on demand. A second tier of latches is provided, the second tier having multiple second tier latches to serialize access to corresponding allocated sequences of values such that at any point in time, only one of the concurrently executing session processes is granted access to the allocated sequence. The multiple tiers serve to reduce the likelihood of contention to the first tier latch. | 06-06-2013 |
20130151567 | TECHNIQUES FOR MORE EFFICIENT USAGE OF MEMORY-TO-CPU BANDWIDTH - Techniques are provided for more efficiently using the bandwidth of the I/O path between a CPU and volatile memory during the performance of database operation. Relational data from a relational table is stored in volatile memory as column vectors, where each column vector contains values for a particular column of the table. A binary-comparable format may be used to represent each value within a column vector, regardless of the data type associated with the column. The column vectors may be compressed and/or encoded while in volatile memory, and decompressed/decoded on-the-fly within the CPU. Alternatively, the CPU may be designed to perform operations directly on the compressed and/or encoded column vector data. In addition, techniques are described that enable the CPU to perform vector processing operations on the column vector values. | 06-13-2013 |
20130151568 | TECHNIQUES FOR MAINTAINING COLUMN VECTORS OF RELATIONAL DATA WITHIN VOLATILE MEMORY - Techniques are provided for more efficiently using the bandwidth of the I/O path between a CPU and volatile memory during the performance of database operation. Relational data from a relational table is stored in volatile memory as column vectors, where each column vector contains values for a particular column of the table. A binary-comparable format may be used to represent each value within a column vector, regardless of the data type associated with the column. The column vectors may be compressed and/or encoded while in volatile memory, and decompressed/decoded on-the-fly within the CPU. Alternatively, the CPU may be designed to perform operations directly on the compressed and/or encoded column vector data. In addition, techniques are described that enable the CPU to perform vector processing operations on the column vector values. | 06-13-2013 |
20130159246 | ON-LINE TRANSACTION PROCESSING (OLTP) COMPRESSION AND RE-COMPRESSION OF DATABASE DATA - A computer is programmed to compress data of a database in response to database modification language (DML) statements generated by on-line transaction processing (OLTP) systems. In several embodiments, data that is initially added to a database block is left uncompressed until a predetermined condition is satisfied, which happens infrequently (relative to OLTP transactions on the block). When satisfied, the computer automatically compresses all uncompressed data in the block, which increases the amount of unused space in the block. New data is thereafter added uncompressed to the partially compressed block, until satisfaction of a predetermined condition whereby the partially compressed block is again compressed, i.e. re-compressed. Adding of new data to a partially compressed block and its compression are repeated unless another predetermined condition is met, in response to which the block is not further re-compressed, thereby to recognize a limit on the benefit from compression. | 06-20-2013 |
20130275473 | TECHNIQUES FOR MAINTAINING COLUMN VECTORS OF RELATIONAL DATA WITHIN VOLATILE MEMORY - Techniques are provided for more efficiently using the bandwidth of the I/O path between a CPU and volatile memory during the performance of database operation. Relational data from a relational table is stored in volatile memory as column vectors, where each column vector contains values for a particular column of the table. A binary-comparable format may be used to represent each value within a column vector, regardless of the data type associated with the column. The column vectors may be compressed and/or encoded while in volatile memory, and decompressed/decoded on-the-fly within the CPU. Alternatively, the CPU may be designed to perform operations directly on the compressed and/or encoded column vector data. In addition, techniques are described that enable the CPU to perform vector processing operations on the column vector values. | 10-17-2013 |
20140013076 | EFFICIENT HARDWARE INSTRUCTIONS FOR SINGLE INSTRUCTION MULTIPLE DATA PROCESSORS - A method and apparatus for efficiently processing data in various formats in a single instruction multiple data (“SIMD”) architecture is presented. Specifically, a method to unpack a fixed-width bit values in a bit stream to a fixed width byte stream in a SIMD architecture is presented. A method to unpack variable-length byte packed values in a byte stream in a SIMD architecture is presented. A method to decompress a run length encoded compressed bit-vector in a SIMD architecture is presented. A method to return the offset of each bit set to one in a bit-vector in a SIMD architecture is presented. A method to fetch bits from a bit-vector at specified offsets relative to a base in a SIMD architecture is presented. A method to compare values stored in two SIMD registers is presented. | 01-09-2014 |
20140013077 | EFFICIENT HARDWARE INSTRUCTIONS FOR SINGLE INSTRUCTION MULTIPLE DATA PROCESSORS - A method and apparatus for efficiently processing data in various formats in a single instruction multiple data (“SIMD”) architecture is presented. Specifically, a method to unpack a fixed-width bit values in a bit stream to a fixed width byte stream in a SIMD architecture is presented. A method to unpack variable-length byte packed values in a byte stream in a SIMD architecture is presented. A method to decompress a run length encoded compressed bit-vector in a SIMD architecture is presented. A method to return the offset of each bit set to one in a bit-vector in a SIMD architecture is presented. A method to fetch bits from a bit-vector at specified offsets relative to a base in a SIMD architecture is presented. A method to compare values stored in two SIMD registers is presented. | 01-09-2014 |
20140013078 | EFFICIENT HARDWARE INSTRUCTIONS FOR SINGLE INSTRUCTION MULTIPLE DATA PROCESSORS - A method and apparatus for efficiently processing data in various formats in a single instruction multiple data (“SIMD”) architecture is presented. Specifically, a method to unpack a fixed-width bit values in a bit stream to a fixed width byte stream in a SIMD architecture is presented. A method to unpack variable-length byte packed values in a byte stream in a SIMD architecture is presented. A method to decompress a run length encoded compressed bit-vector in a SIMD architecture is presented. A method to return the offset of each bit set to one in a bit-vector in a SIMD architecture is presented. A method to fetch bits from a bit-vector at specified offsets relative to a base in a SIMD architecture is presented. A method to compare values stored in two SIMD registers is presented. | 01-09-2014 |
20140067767 | GENERATING AND APPLYING REDO RECORDS - Techniques for maintaining a cascading index are provided. In one approach, one or more branch node compression techniques are applied to the main index of a cascading index. In an approach, a Bloom filter is generated and associated with, e.g., a branch node in the main index. The Bloom filter is used to determine whether, without accessing any leaf blocks, a particular key value exists, e.g., in leaf blocks associated with the branch node. In an approach, a new redo record is generated in response to a merge operation between two levels of the cascading index. The new redo record comprises (a) one or more addresses of blocks that are affected by the merge operation, (b) data is that being “pushed down” to a lower level of the cascading index, and (c) one or more addresses of blocks that are written to storage as a result of the merge operation. | 03-06-2014 |
20140074805 | STORING COMPRESSION UNITS IN RELATIONAL TABLES - A database server stores compressed units in data blocks of a database. A table (or data from a plurality of rows thereof) is first compressed into a “compression unit” using any of a wide variety of compression techniques. The compression unit is then stored in one or more data block rows across one or more data blocks. As a result, a single data block row may comprise compressed data for a plurality of table rows, as encoded within the compression unit. Storage of compression units in data blocks maintains compatibility with existing data block-based databases, thus allowing the use of compression units in preexisting databases without modification to the underlying format of the database. The compression units may, for example, co-exist with uncompressed tables. Various techniques allow a database server to optimize access to data in the compression unit, so that the compression is virtually transparent to the user. | 03-13-2014 |
20140095438 | TRACKING ROW AND OBJECT DATABASE ACTIVITY INTO BLOCK LEVEL HEATMAPS - A method, apparatus, and system for tracking row and object database activity into block level heatmaps is provided. Database activity including reads, writes, and creates can be tracked by a database management system at the finest possible level of granularity, or the row and object level. To efficiently record the tracked database activity, a two-part structure is described for writing the activity into heatmaps. A hierarchical in-memory component may use a dynamically allocated sparse pool of bitmap blocks. Periodically, the in-memory component is persisted to a stored representation component, sharable with multiple database instances, which may include consolidated last access times and/or a history of heatmap snapshots to reflect access over time. The heatmaps may then be externalized to database users and applications to provide and support a variety of features. | 04-03-2014 |
20140095442 | TECHNIQUES FOR LIFECYCLE STATE MANAGEMENT AND IN-DATABASE ARCHIVING - Techniques for lifecycle state management and in-database archiving are described. Activity tracking refers to techniques that collect statistics related to user access patterns, such as the frequency or recency with which users access particular database elements. The statistics gathered through activity tracking can be supplied to data classification techniques to automatically classify the database elements or to assist users with manually classifying the database elements. Then, once the database elements have been classified, in-database archiving techniques can be employed to move database elements to different storage tiers based on the classifications. However, although the techniques related to activity tracking, data classification, and in-database archiving may be used together as described above; each technique may also be practiced separately. | 04-03-2014 |
20140095448 | AUTOMATED INFORMATION LIFECYCLE MANAGEMENT USING LOW ACCESS PATTERNS - A method, apparatus, and system for automated information lifecycle management using low access patterns in a database management system are provided. A user or the database can store policy data that defines an archiving action when meeting an activity-level condition on one or more database objects. The archiving actions may include compression, data movement, and other actions to place the database object in an appropriate storage tier for a lifecycle phase of the database object. The activity-level condition may specify the database object meeting a low access pattern, optionally for a minimum time period. Various criteria including access statistics for the database object and cost characteristics of current and target compression levels or storage tiers may be considered to determine the meeting of the activity-level condition. The policies may be evaluated on an adjustable periodic basis and may utilize a task scheduler for minimal performance impact. | 04-03-2014 |
20140095449 | Policy Driven Data Placement And Information Lifecycle Management - A method, apparatus, and system for policy driven data placement and information lifecycle management in a database management system are provided. A user or database application can specify declarative policies that define the movement and transformation of stored database objects. The policies are associated with a database object and may also be inherited. A policy defines, for a database object, an archiving action to be taken, a scope, and a condition before the archiving action is triggered. Archiving actions may include compression, data movement, table clustering, and other actions to place the database object into an appropriate storage tier for a lifecycle phase of the database object. Conditions based on access statistics can be specified at the row level and may use segment or block level heatmaps. Policy evaluation occurs periodically in the background, with actions queued as tasks for a task scheduler. | 04-03-2014 |
20140095450 | Custom Policy Driven Data Placement And Information Lifecycle Management - A method, apparatus, and system for custom policy driven data placement and information lifecycle management in a database management system are provided. A user or database application can specify declarative custom policies that define the movement and transformation of stored database objects. A custom policy defines, for a database object, a custom function to evaluate on an associated database object to determine whether an archiving action is triggered. Archiving actions may include compression, data movement, table clustering, and other actions to place the database object into an appropriate storage tier for a lifecycle phase of the database object. The custom function is defined by the database user, and can flexibly include any customized business logic using data sources internal and external to the database, including database access statistics such as segment level or block level heatmaps. | 04-03-2014 |
20140095517 | PREDICATE RESULT CACHE - In token compression, a column value is represented by a encoded token. During early predicate evaluation, the result that a decoded token in a column does not satisfy a predicate condition in a predicate conjunction is recorded in a cache as a “FALSE”. Such a result not only means that when the column contains that token that the predicate condition is not satisfied, but also that a predicate conjunction containing the predicate condition is not satisfied. When performing early predicate evaluation on a subsequent row on a token in a column, the conjunction result cache is referred to. If the cache records a FALSE for that token in that column, then it is known the predicate conjunction cannot be satisfied by the decoded token. | 04-03-2014 |
20140095547 | TECHNIQUES FOR ACTIVITY TRACKING, DATA CLASSIFICATION, AND IN DATABASE ARCHIVING - Techniques for activity tracking, data classification, and in-database archiving are described. Activity tracking refers to techniques that collect statistics related to user access patterns, such as the frequency or recency with which users access particular database elements. The statistics gathered through activity tracking can be supplied to data classification techniques to automatically classify the database elements or to assist users with manually classifying the database elements. Then, once the database elements have been classified, in-database archiving techniques can be employed to move database elements to different storage tiers based on the classifications. However, although the techniques related to activity tracking, data classification, and in-database archiving may be used together as described above; each technique may also be practiced separately. | 04-03-2014 |
20140279849 | HIERARCHICAL TABLESPACE SPACE MANAGEMENT - A method, apparatus, and system for hierarchical organization of tablespace free space metadata in a database are provided. The hierarchy is divided into at least two levels: level 1 or L1 bitmap blocks are stored as a shared data structure and map free space in a tablespace, whereas level 2 or L2 bitmap blocks are stored as separate local copies at each database instance and map to the L1 bitmap blocks. This hierarchical organization provides a mechanism for finer grained concurrency control, enabling highly parallel tablespace metadata processing to accommodate the performance requirements of large tablespaces with big data sets. By integrating the hierarchical organization as part of the tablespace stack layer in a database management system (DBMS), implementations can be provided transparently to database end users without demanding any additional administrative, maintenance, or development burdens. | 09-18-2014 |
20140279959 | OLTP COMPRESSION OF WIDE TABLES - A data block stores one or more rows of a database table or relation. An entire row may not fit in a data block. Part of the row is stored in one data block, and another part is stored in another data block. Each row part is referred to herein as a row segment and the data blocks are referred to as row-segmented data blocks. Data block dictionary compression is used to compress row-segmented data blocks. Each data block contains a dictionary that is used to compress rows in the data block, including row segments. The dictionary in a data block is used to compress row segments in the data block. Hence, multiple dictionaries may be used to decompress a row comprised of row segments. | 09-18-2014 |
20140279960 | Row Level Locking For Columnar Data - Row locking is performed at the row level of granularity for database data stored in columnar form. Row level locking entails use of a lock vector that is stored in a compression unit in a data block, the compression unit storing rows in columnar-major format. On an as needed basis, the lock vector is expanded to identify more transactions affecting the rows in the compression unit. | 09-18-2014 |
20140280249 | PREDICATE OFFLOAD OF LARGE OBJECTS - In an approach, the database server generates a request for data representing rows of a table, the request identifying one or more data blocks stored on a storage system. The database server then generates metadata describing one or more filtering conditions to be applied to the rows and sends the metadata along with the request to the storage system. The storage system, when applying filtering conditions to a column containing a LOB, determines whether the LOB is stored in-line or out-of-line. If the column contains an out-of-line LOB, the storage system skips the filtering conditions on the column. If the column contains an in-line LOB, the storage system applies the filtering conditions to the column. Upon obtaining the filtered data from the storage system, the database server retrieves the data blocks for out-of-line LOBs and applies the skipped filtering conditions to create a final result set. | 09-18-2014 |
20140281247 | METHOD TO ACCELERATE QUERIES USING DYNAMICALLY GENERATED ALTERNATE DATA FORMATS IN FLASH CACHE - A method for accelerating queries using dynamically generated columnar data in a flash cache is provided. In an embodiment, a method comprises a storage device receiving a first request for data that is stored in the storage device in a base major format in one or more primary storage devices. The storage device comprises a cache. The base major format is any one of: a row-major format, a column-major format and a hybrid-columnar format. Based on first one or more criteria, it is determined whether to rewrite the data into rewritten data in a rewritten major format. In response to determining to rewrite the data into rewritten data in a rewritten major format, the storage device rewrites at least a portion of the data into particular rewritten data in the rewritten major format. The rewritten data is stored in the cache. | 09-18-2014 |
20150032763 | QUERY AND EXADATA SUPPORT FOR HYBRID COLUMNAR COMPRESSED DATA - A method and apparatus is provided for optimizing queries received by a database system that relies on an intelligent data storage server to manage storage for the database system. Storing compression units in hybrid columnar format, the storage manager evaluates simple predicates and only returns data blocks containing rows that satisfy those predicates. The returned data blocks are not necessarily stored persistently on disk. That is, the storage manager is not limited to returning disc block images. The hybrid columnar format enables optimizations that provide better performance when processing typical database workloads including both fetching rows by identifier and performing table scans. | 01-29-2015 |
20150088811 | MECHANISM TO RUN OLTP WORKLOAD ON IN-MEMORY DATABASE UNDER MEMORY PRESSURE - Techniques are provided for maintaining data persistently in one format, but making that data available to a database server in more than one format. For example, one of the formats in which the data is made available for query processing is based on the on-disk format, while another of the formats in which the data is made available for query processing is independent of the on-disk format. Data that is in the format that is independent of the disk format may be maintained exclusively in volatile memory to reduce the overhead associated with keeping the data in sync with the on-disk format copies of the data. | 03-26-2015 |
20150088822 | MULTI-VERSION CONCURRENCY CONTROL ON IN-MEMORY SNAPSHOT STORE OF ORACLE IN-MEMORY DATABASE - Techniques are provided for maintaining data persistently in one format, but making that data available to a database server in more than one format. For example, one of the formats in which the data is made available for query processing is based on the on-disk format, while another of the formats in which the data is made available for query processing is independent of the on-disk format. Data that is in the format that is independent of the disk format may be maintained exclusively in volatile memory to reduce the overhead associated with keeping the data in sync with the on-disk format copies of the data. | 03-26-2015 |
20150088824 | GRANULAR CREATION AND REFRESH OF COLUMNAR DATA - Techniques are provided for granular load and refresh of columnar data. In an embodiment, a particular data object that contains particular data formatted different from column-major format is maintained, the particular data including first data and second data. First and second data objects contain the first and second data, respectively, organized in the column-major format. In response to changes being committed to the first data in the particular data object, invalidating one or more rows of the first data object. In response to a number of invalidated rows of the first data object exceeding a threshold, automatically performing a refresh operation on the first data object independent of any refresh operation on the second data object. | 03-26-2015 |
20150088830 | MIRRORING, IN MEMORY, DATA FROM DISK TO IMPROVE QUERY PERFORMANCE - Techniques are provided for maintaining data persistently in one format, but making that data available to a database server in more than one format. For example, one of the formats in which the data is made available for query processing is based on the on-disk format, while another of the formats in which the data is made available for query processing is independent of the on-disk format. Data that is in the format that is independent of the disk format may be maintained exclusively in volatile memory to reduce the overhead associated with keeping the data in sync with the on-disk format copies of the data. | 03-26-2015 |
20150089125 | FRAMEWORK FOR NUMA AFFINITIZED PARALLEL QUERY ON IN-MEMORY OBJECTS WITHIN THE RDBMS - Techniques are provided for performing parallel processing on in-memory objects within a database system. In one embodiment, a plurality of in-memory chunks are maintained on a plurality of non-uniform memory access (NUMA) nodes. In response to receiving a query, a set of clusters is determined for the plurality of in-memory chunks. Each respective cluster in the set of clusters corresponds to a particular NUMA node of the plurality of NUMA nodes and includes a set of one or more in-memory chunks from the plurality of in-memory chunks. For each respective cluster in the set of clusters, a query coordinator assigns, to the respective cluster, a set of one or more processes associated with the particular NUMA node that corresponds to the respective cluster. | 03-26-2015 |
20150089134 | CORE IN-MEMORY SPACE AND OBJECT MANAGEMENT ARCHITECTURE IN A TRADITIONAL RDBMS SUPPORTING DW AND OLTP APPLICATIONS - Techniques are provided for managing in-memory space and objects. In one embodiment, a set of in-memory objects are maintained within an area in volatile memory that is accessible to a database server. An in-memory object in this context includes a set of one or more in-memory segments where each respective in-memory segment includes a set of in-memory extents and each respective in-memory extent is a contiguous chunk of memory from the area in volatile memory that is accessible to the database server. The area in volatile memory is managed as a set of stripes, where each stripe is a contiguous chunk of in-memory extents. Stripe control blocks are used to locate free in-memory extents for allocation and registration with an in-memory segment. | 03-26-2015 |