CHANGELOG

1.7.1

1.7.0
- fix: welcome banner
- code style cleanup
- fix: diagnostic information populated when not enabled
- enhancement: add dictionary values for TIMEZONE database parameter
- enhancement: add parameter -r|--root to disable root check on startup
- fix: in rare cases column value is null instead of actual value
- fix: when redo log file is truncated, read error might appear
- documentation update
- fix: missing include for some systems
- enhancement: thread performance information
- enhancement: swap memory to disk in low memory situations

1.6.0
- enhancement: add parameter -p
- enhancement: performance optimizations
- enhancement: add offset to payload for debugging purposes
- enhancement: support UPDATE operations for NOT NULL columns with occasional NULL values (experimental)
- enhancement: add JSON validation for invalid tags
- fix: context field contains invalid value
- fix: invalid tag in example json config file
- fix: online redo log are read in rare cases for archived redo log only mode
- fix: don't write checkpoint file until the first data chunk is processed
- fix: minor bug related to crash with large number of columns
- fix: error: "parse error, attribute xid: not expected"
- fix: in rare cases, the AFTER value for UPDATE is missing
- code style cleanup
- documentation update
- fix: welcome message

1.5.0
- code style cleanup
- enhancement: new writer type: discard
- minor code optimizations
- fix: minor bug related to binary xmltype
- fix: correctly display Oracle error codes
- fix: static code analyzer warnings
- fix: checkpointing is not working for online mode after restart
- enhancement: add an option to build statically linked binary
- fix: don't require flag 64 to use experimental XMLTYPE support
- fix: minor bugs related to timestamp formatting
- minor documentation fixes
- enhancement: refactor timestamps, add more output options
- enhancement: collecting metrics using Prometheus
- configuration change: moved memory options "memory-max-mb", "memory-min-mb", "read-buffer-max-mb" to "memory" group and renamed to "max-mb", "min-mb"

1.4.0
- fix: date and timestamp with year before 1900 or after 2262 produce incorrect values
- enhancement: unify gencfg.sql script for 11g and 12+
- fix: bad checkpoint file causes startup to fail with no explanation
- fix: fail with a too short field length message
- code style cleanup
- fix: LOB data handling
- enhancement: add session information to output
- enhancement: add row filtering based on condition
- experimental: support for binary xmltype, added document about experimental features
- enhancement: added support for BOOLEAN data type
- enhancement: added support for Oracle 23c
- enhancement: Oracle 23c: set column limit to 4096
- clang-tidy code reformatting

1.3.0
- doc: updated README.md, added reference to a tutorial GitHub project
- fix: PDB/CDB support
- fix: unified comment style
- fix: checkpoint file is not created when no data is replicated after start
- fix: offline startup failed to read redo log files
- fix: DDL operation that rearranges columns is not handled properly
- enhancement: reload config file for schema update after the file is updated
- fix: stacktrace when processing DDL data to protobuf format
- fix: typo in trace message
- enhancement: added more timestamp formatting options
- enhancement: added support for UROWID format
- fix: LOB handing improvement for Oracle 19c
- enhancement: added timezone field for timestamp with a timezone type
- fix: file writer path handling and file handling fixes
- enhancement: add database name to payload data
- enhancement: add schema information for ddl events
- fix: correctly handling of non-ASCII characters in DDL text
- enhancement: added support for an interval type
- enhancement: added xmltype support
- enhancement: added support for type timestamp with local time zone
- fix: client stream disconnect causes replication to stop
- enhancement: added options interval-ytm and interval-dts to control interval type output format
- fix: minor style fixes
- fix: network client date was not consistent when reconnected
- fix: scn informatio
n inconsistency in transaction fix
- enhancement: added flag 2 for "scn" field type to control scn values
- enhancement: changed checkpointing mechanism to allow more frequent checkpoints
- experimental: added partial support for binary XMLType data
- enhancement: performance optimizations

1.2.0
- doc: Introduction to OpenLogReplicator
- fix: minor bug related to sending ddl operations to output
- fix: minor parameter validation bug
- fix: minor issue related to sending character data in hex format for debug purposes
- cleanup in parameter names: renamed "max-size" "max-file-size" and "format" to "timestamp-format" for better understanding
- doc: Commands Reference Manual
- doc: Log Messages Guide
- doc: Installation Guide
- doc: Release Notes
- doc: Troubleshooting Guide
- doc: User Manual
- fix: treat character decoding issues as warnings
- fix: removed obsolete example file
- fix: removed obsolete files
- changed "trace" parameter to "log-level"
- changed "trace2" parameter to "trace"
- fix: unification of log messages
- change: config file version is "1.2.0"
- fix: changed type of nullable field from numeric to boolean
- fix: uncaught exception in writer module
- fix: protocol buffer message missing scn data
- fix: allow server to handle bad start requests for network mode
- fix: network client display fixes and handle big messages
- fix: network client memory release on error
- change: removed "max-messages", "enable-idempotence" and "brokers" parameters and added "properties" parameter instead
- change: by default checkpoint messages are not sent to output
- fix: compilation error when Kafka is disabled
- fix: checkpoint did not advance when no data was replicated

1.1.0
- debug: trace formatting changes
- enhancement: add handling of OP 26.2 for experimental LOB support
- fix: experimental LOB support
- enhancement: code style corrections
- fix: error handling of incorrect config file
- fix: experimental LOB support Unicode character decoding
- new feature: basicfile LOB (removed being experimental)

1.0.1 - released on 2023.02.20
- fix: incorrect checkpoint file when non-ascii characters are used in schema files
- fix: incorrect message printed on error
- fix: allow to use stop-* parameters without using schemaless mode
- enhancement: added consistency checks for experimental LOB support
- fix: standardization of output messages
- fix: race condition might result in transaction data loss
- fix: experimental LOB support schema script error
- enhancement: optimized DDL processing speed
- enhancement: added debug info for release builds
- enhancement: added support for Oracle 19.18 and 21.8
- fix: experimental LOB data missing

1.0.0 - released on 2022.12.10
- copy from 0.9.58

0.9.58
- added support for Oracle 19.17 and 21.7
- bug fixes
- code cleanup

0.9.57
- fixed compilation warnings for Release

0.9.56
- bug fixes

0.9.55
- experimental lob 12.1+ fixes
- fixed bugs related to schema & checkpoints

0.9.54
- bug fixes

0.9.53
- bug fixes

0.9.52
- experimental LOB processing fixes
- minor bug fixes
- code style cleanup

0.9.51
- added support for binary_float, binary_double types

0.9.50
- big fixes & code rewrite
- added experimental support for LOBs
- added support for Oracle 19.15 and 21.6
- added "dump-xid" parameter to config file

0.9.49
- small fixes

0.9.48
- fixed old checkpoints deletion

0.9.47
- small fixes

0.9.46
- small thread fixes

0.9.45
- small fixes

0.9.44
- fixed compilation errors for Release target
- small fixes, minor code cleanup

0.9.43
- fixed compilation errors for Release target
- code style fixes
- minor fixes
- fixed issues related to standby database handling

0.9.42
- moved from Makefile to Cmake
- major code refactor, removed spaghetti code, divided code into separate modules, name cleanup, style cleanup, etc.
- fixed and extended logdump output
- fixed minor issues
- updated config file format
- added new parameter: "keep-checkpoints"
- removed experimental not working code: ASM support, RocketMQ, Redis"
- changed "debug"."flush-buffer" to "debug"."flush-on-exit" in json config
- fixed pre-1970 timestamp values in Unix format

0.9.41-beta
- conversion bug fixes

0.9.40-beta
- performance optimizations

0.9.39-beta
- fixed some logdump output
- fixed bug: displayed warning about kteop field size
- added support for Oracle 19.14 and 21.5
- big code cleanup and rewrite of some parts
- fixed bug: split undo record + rollback was not properly handled
- fixed bug: sometimes supplemental redo log was missed
- fixed bug: sometimes partially rolled transactions were improperly handled
- changed Oracle client to 19.14
- cleanup in file access flags

0.9.38-beta
- minor code cleanup
- changed Oracle client to 19.13
- bumped up year
- fixed some logdump output
- fixed bug: batch mode failed

0.9.37-beta
- code cleanup - removed default namespaces
- fixed bug: reading failed when switch to standby was run

0.9.36-beta
- fixed Kafka compilation errors
- fixed Protobuf compilation errors

0.9.35-beta
- fixed bug: throwing exception in constructor [#28]
- fixed bug: throwing exception in callback functions for Kafka module
- fixed bug: error "split undo is broken" when undo for op code 11.22 split
- fixed bug: reading from standby redo log files failed during switch
- optimized Oracle connection handling

0.9.34-beta
- fixed bug: reading from standby database
- removed distinct config for standby database: "online-standby" is not required, use "online" instead

0.9.33-beta
- fixed bug: throwing unhandled exception from callback functions
- fixed bug: after resetlogs starting redo log number might be incorrect
- fixed bug: stacktrace when cancelled and no database connection
- fixed bug: database server restart was incorrectly handled
- fixed bug: forced checkpoint files creation in case of log switch
- OpenLogReplicator.json: updated format version to 0.9.33
- OpenLogReplicator.json: changed "checkpoint" to "state"
- OpenLogReplicator.json: changed "all" to "all-checkpoints" in "state"
- OpenLogReplicator.json: added option "type" in "state"
- forced creation of checkpoint files for redo log switch and some other conditions
- optimized schema parameters read operation
- added support for Oracle 19.13.0, 21.4.0
- added support for database resetlogs operation

0.9.32-beta
- changed LWN buffers from 512 to 1024 to avoid error "[ERROR] all 512 LWN buffers allocated"

0.9.31-beta
- welcome message includes list of compiled modules
- minor message fixes
- removed collecting SYS.SEG$ from DDL track code
- fixed bug: multiplication of redo log files in schema file
- fixed bug: "flags" parameter value verification
- minor changes in gencfg.sql
- OpenLogReplicator.json: updated format version to 0.9.30
- OpenLogReplicator.json: added new parameter: "dump-path"

0.9.30-beta
- added target: Apache RocketMQ (experimental)
- OpenLogReplicator.json: updated format version to 0.9.30

0.9.29-beta
- fixed bug: "disable-checks" parameter value check
- fixed bug: schemaless mode failing with compressed tables
- fixed bug: incorrect string offset in checkpoint file
- fixed bug: reading from inactive online redo log could print warnings/errors
- fixed bug: merging split undo blocks could sometimes fail
- fixed bug: schemaless processing failed with DML commands
- fixed bug: direct read mode was not working
- fixed bug: null values where added for unknown values
- added content checks for redo decoder/column integrity
- added support for unused columns ("flags" parameter have shifted)
- OpenLogReplicator.json: default for "redo-verify-delay-us" is 0 when direct io is used, else 500000
- minor code cleanups

0.9.28-beta
- fixed checkpointing on instrumented shutdown
- removed LWN checking from version 0.9.27-beta

0.9.27-beta
- added verification checks for LWN header
- changed "sequence" to "seq" in checkpoint file - you might need to update manually json checkpoint files
- fixed bug: incorrect con length read from schema file
- added some tracking debug info
- added check for ignoring binary objects (like xmltype, object)
- removed debug table from output (to make testing easier)
- removed warning for 11.2 during ALTER TABLE ADD COLUMN with DEFAULT
- fixed bugs related to checkpoints
- fixed bug: sending checkpoint messages for already confirmed SCN

0.9.26-beta
- fixed memory leaks during abnormal exit
- added content validation for checkpoint files
- added length check for reading JSON config and checkpoint files
- added minor performance optimizations for JSON output

0.9.25-beta
- fixed bugs in gencfg.sql
- added checks for block size in redo header
- added consistency checks for schemas
- OpenLogReplicator.json: renamed parameter "arch-read-retry" to "arch-read-tries"
- OpenLogReplicator.json: updated format version to 0.9.25

0.9.24-beta
- updated automake to version 1.16.3
- fixed typo bugs in example JSON files
- display performance decoder also for online redo logs ("trace2": 256)
- fixed null values for mode "column": 1, 2
- enhanced read error messages
- updated README.md

0.9.23-beta
- hidden constraint value for nested tables
- fixed bug: in idle used 100% CPU
- quick fix for skipping compressed rows (experimental)
- OpenLogReplicator.json: changed default for "redo-read-sleep-us" parameter from 10000 to 50000

0.9.22-beta
- added support for Oracle 21c (21.3)
- OpenLogReplicator.json: field length limit for broker is 4096, topic is 256, context is 30, database to 256

0.9.21-beta
- added length checks for string types
- minor code cleanup & optimization to reduce memory allocations

0.9.20-beta
- changed Oracle client to 19.12
- build scripts compiled with newer autotools
- fixed bug with output file path for json decoder

0.9.19-beta
- added support for Oracle 19.12.0
- added stacktrace print on signal USR1

0.9.18-beta
- fixed bug: scn could become 0 in checkpoint record when redo log files was empty (had just 2 pages)
- NetworkClient - fixed bug with memory leak on abnormal exit
- fixed memory leak for network library
- fixed bug in Protobuf output module

0.9.17-beta
- code cleanup to be become more compliant
- fixed memory leak on abnormal exit
- fixed startup based on checkpoint decoder
- fixed schema tracking after restart
- fixed bug: sometimes log switch checkpoint was missing
- minor code cleanup & fixes
- OpenLogReplicator.json: updated format version to 0.9.16
- OpenLogReplicator.json: enhanced content validation, fixed and extended syntax checking
- OpenLogReplicator.json: increased default of redo-verify-delay-us to 250000
- OpenLogReplicator.json: renamed "sources" to "source"
- OpenLogReplicator.json: renamed "targets" to "target"
- OpenLogReplicator.json: renamed "redo-logs" to "redo-log"
- OpenLogReplicator.json: protobuf/json output: added optional payload numbering ("message" = 2)
- OpenLogReplicator.json: optional begin transaction for output ("message" = 4)
- OpenLogReplicator.json: optional commit transaction for output ("message" = 8)
- OpenLogReplicator.json: new element in "source": "filter"
- OpenLogReplicator.json: moved "tables" to "filter" and renamed to "table"
- OpenLogReplicator.json: added transaction filtering based on XID ("skip-xid")
- OpenLogReplicator.json: added transaction filtering based on size ("transaction-max-mb")

0.9.16-beta
- added check of schema file for changing list of owners of replicated tables
- added check for schema/table name entered in upper case
- minor fixes

0.9.15-beta
- version is no longer used in schema file, instead is used config schema version, currently "0.9.15"
- removed call OCITerminate on exit which caused memory leak from OCI
- added information about schema dropped schema for dropped (or altered) tables
- added check to prevent run as root

0.9.14-beta
- fixed bug: sometimes hanged during shutdown while running OCI
- fixed bug: regex might used improperly
- added check for GCC 4.9
- changed run argument, use -f for specifying custom file

0.9.13-beta
- updated example files names
- fixed JSON output for DDL commands

0.9.12-beta
- fixed bug, when schema file is present reading from archived redo log cause stacktrace

0.9.11-beta
- fixed protobuf checkpoint code (#23)
- fixed sending output to stdout instead of file for "file" mode
- fixed bug for merged values
- fixed json format for schemaless mode

0.9.10-beta
- change "name" to "output" for writer file mode
- added "max-size", "new-line", "format", "append" schemaElements to writer file mode
- file output can rotate output file with defined max size
- internally changed identifier 0 value to -1 for file descriptor
- fixed DDL tracking for some commands (alter table modify column, alter table add subpartition, etc.)
- fix creating log groups multiplication in schema file
- added displaying error information for errno code
- added disk read retry

0.9.9-beta
- fixed subpartition schema load/save
- fixed DDL tracking for columns
- added (sub)partition count for schema list

0.9.8-beta
- fixed compilation errors when g++ is used

0.9.7-beta
- fixed small bug with passing config OpenLogReplicator.json path at startup

0.9.6-beta
- added checks for OpenLogReplicator.json content
- fixed compilation errors when newer gcc version is used

0.9.5-beta
- objects where SYS.OBJ$ is present but SYS.TAB$ entry missing are ignored
- minor optimizations
- moved config file location: OpenLogReplicator.json to scripts/OpenLogReplicator.json
- moved checkpoint file location: <database>-chkpt.json to checkpoints/<database>-chkpt.json
- ./OpenLogReplicator -v|--version - to print banner and exit
- ./OpenLogReplicator custom_path/config.json - to run with custom config file location

0.9.4-beta
- changed conditions for displaying warning of CRC error
- fixed bug in reader module introduced in version 0.9.3-beta which could lead to core dump

0.9.3-beta
- performance improvements
- changed default disk read buffer size to 32MB

0.9.2-beta
- added parameter "rid" for "format"
- by default row id is not included in the output
- online mode: fixed startup by seq
- updated protobuf output module to be in sync with json code
- rewritten column handling to avoid performance issues due to memory allocation/de-allocation
- decreased contention on locks, internal buffer is flushed less often
- inlining code to create output stream in json format
- inlining some code for red log parsing
- corrected performance measuring throughput

0.9.1-beta
- added scrips/gencfg.sql file - to create schema file for offline mode
- OpenLogReplicator.json: added debug parameters to allow instrumented shutdown during tests: shutdown after reaching defined number of checkpoints, transactions or redo log switches
- OpenLogReplicator.json: flush on shutdown parameter rearrangement
- OpenLogReplicator.json: moved instrumented table to debug section
- protobuf request format: moved seq outside tm_val
- allowed initial start with defined both SEQ & SCN
- minor format changes in JSON schema files
- online mode: added connection check & reconnect in case connection is closed
- online mode: minor SQL fixes
- online mode: fix getting log_archive_format parameter for non-FRA databases
- minor code fixes/enhancements/code cleanup

0.9.0-beta
- initial beta version