Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add topic name requirement format #58

Open
wants to merge 4 commits into
base: master
Choose a base branch
from
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
81 changes: 49 additions & 32 deletions README.md
Original file line number Diff line number Diff line change
@@ -1,5 +1,5 @@

# Kafka Java Client for Oracle Transactional Event Queues
# Kafka Java Client for Oracle Transactional Event Queues

## Building the Kafka Java Client for Oracle TxEventQ distribution

Expand All @@ -9,14 +9,13 @@ You need to have [Gradle 7.3 or above](http://www.gradle.org/installation) and [

This distribution contains version 23.4.0.0 of the `Kafka Java Client for Oracle Transactional Event Queues` project. It will be referred as OKafka-23.4.0.0 henceforth. This is tested with JDK 11.0.22 but we recommend using the latest version.

The Kafka Java Client works with Oracle Database 23ai Free version as well as Oracle Database 23ai available on Oracle Autonomous Cloud platform.
The Kafka Java Client works with Oracle Database 23ai Free version as well as Oracle Database 23ai available on Oracle Autonomous Cloud platform.

To test this distribution in free Oracle Cloud environment create [Oracle Cloud account](https://docs.cloud.oracle.com/en-us/iaas/Content/FreeTier/freetier.htm) then create [Oracle Autonomous Transaction Processing Database instance](https://docs.oracle.com/en/cloud/paas/autonomous-data-warehouse-cloud/tutorial-getting-started-autonomous-db/index.html) in cloud.
To test this distribution in free Oracle Cloud environment create [Oracle Cloud account](https://docs.cloud.oracle.com/en-us/iaas/Content/FreeTier/freetier.htm) then create [Oracle Autonomous Transaction Processing Database instance](https://docs.oracle.com/en/cloud/paas/autonomous-data-warehouse-cloud/tutorial-getting-started-autonomous-db/index.html) in cloud.

A database user should be created and should be granted the privileges mentioned in Database user configuration section. Then create a Transactional Event Queue to produce and consume messages.


### Database user configuration ###
### Database user configuration

To run `OKafka application` against Oracle Database, a database user must be created and must be granted below privileges.

Expand Down Expand Up @@ -46,7 +45,7 @@ It is preferred in general to assign or grant a specific quota on a tablespace t
ALTER USER user QUOTA UNLIMITED /* or size-clause */ on tablespace_name;
```

Once user is created and above privileges are granted, connect to Oracle Database as this user and create a Transactional Event Queue using below PL/SQL script. One can also use `KafkaAdmin` interface as shown in `CreateTopic.java` in `examples` directory to create a Transactional Event Queue.
Once user is created and above privileges are granted, connect to Oracle Database as this user and create a Topic using below PL/SQL script.

```roomsql
-- Create an OKafka topic named 'TXEQ' with 5 partition and retention time of 7 days.
Expand All @@ -55,50 +54,69 @@ begin
end;
```

#### Connection configuration ####
> Note: A Topic can also be created using OKAFKA Administration methods. Or, through the Producer interface which creates a new topic if it was not previously created.

#### Connection configuration

`OKafka` uses JDBC(thin driver) connection to connect to Oracle Database instance using any one of two security protocols.

1. PLAINTEXT
2. SSL

1. PLAINTEXT
2. SSL

1.PLAINTEXT: In this protocol a JDBC connection is setup by providing username and password in plain text in ojdbc.prperties file. To use PLAINTEXT protocol user must provide following properties through application.

security.protocol = "PLAINTEXT"
bootstrap.servers = "host:port"
oracle.service.name = "name of the service running on the instance"
oracle.net.tns_admin = "location of ojdbc.properties file"

```text
security.protocol = "PLAINTEXT"
bootstrap.servers = "host:port"
oracle.service.name = "name of the service running on the instance"
oracle.net.tns_admin = "location of ojdbc.properties file"
```

`ojdbc.properties` file must have below properties

user(in lowercase)=DatabaseUserName
password(in lowercase)=Password

```text
user(in lowercase)=DatabaseUserName
password(in lowercase)=Password
```

2.SSL: This protocol requires that, while connecting to Oracle Database, the JDBC driver authenticates database user using Oracle Wallet or Java KeyStore(JKS) files. This protocol is typically used to o connect to Oracle database 23ai instance in Oracle Autonomous cloud. To use this protocol `Okafka` application must specify following properties.

security.protocol = "SSL"
oracle.net.tns_admin = "location containing Oracle Wallet, tnsname.ora and ojdbc.properties file"
tns.alias = "alias of connection string in tnsnames.ora"
```text
security.protocol = "SSL"
oracle.net.tns_admin = "location containing Oracle Wallet, tnsname.ora and ojdbc.properties file"
tns.alias = "alias of connection string in tnsnames.ora"
```

Directory location provided in `oracle.net.tns_admin` property should have:

Directory location provided in `oracle.net.tns_admin` property should have
1. Oracle Wallet
2. tnsnames.ora file
3. ojdbc.properties file (optional)
This depends on how the Oracle Wallet is configured.

Learn more about [JDBC Thin Connections with a Wallet (mTLS)](https://docs.oracle.com/en/cloud/paas/atp-cloud/atpug/connect-jdbc-thin-wallet.html#GUID-5ED3C08C-1A84-4E5A-B07A-A5114951AA9E) to establish secured JDBC connections.

Note: tnsnames.ora file in wallet downloaded from Oracle Autonomous Database contains JDBC connection string which is used for establishing JDBC connection.


> Note: tnsnames.ora file in wallet downloaded from Oracle Autonomous Database contains JDBC connection string which is used for establishing JDBC connection.

#### APIs configuration

You can get a detailed description of the Producer, Consumer and Administration APIs in the [Kafka APIs for Oracle Transactional Event Queues Documentation](https://docs.oracle.com/en/database/oracle/oracle-database/23/adque/Kafka_cient_interface_TEQ.html#GUID-5549915E-6509-4065-B05E-E96338F4742C).

> Note: Topic name property should be provided in UPPERCASE.
>
>> ```text
>> topic.name=<Oracle Database TxEventQ Topic, use uppercase>
>> ```

### Building okafka.jar

Simplest way to build the `okafka.jar` file is by using Gradle build tool.
This distribution contains gradle build files which will work for Gradle 7.3 or higher.

```
```shell
./gradle jar
```

This generates `okafka-23.4.0.0.jar` in `okafka_source_dir/clients/build/libs`.

**Project Dependency:**
Expand All @@ -119,18 +137,17 @@ All these jars are downloaded from Maven Repository during gradle build.

To build the `okafka.jar` file which includes all the dependent jar files in itself.

```
```shell
./gradle fullJar
```
This generates `okafka-full-23.4.0.0.jar` in `okafka_source_dir/clients/build/libs`.


## Build javadoc
This genddsferates `okafka-full-23.4.0.0.jar` in `okafka_source_dir/clients/build/libs`.

## Build javadoc

This command generates javadoc in `okafka_source_dir/clients/build/docs/javadoc`

```
```shell
gradle javadoc
```

Expand All @@ -142,7 +159,7 @@ Repository contains 2 common OKafka application examples in `examples` folder.
Produces 10 messages into TxEQ topic.

`2. ConsumerOKafka.java`
Consumes 10 messages from TxEQ topic.
Consumes 10 messages from TxEQ topic.

## Kafka Java Client APIs supported

Expand Down
25 changes: 25 additions & 0 deletions examples/consumer/src/main/resources/config.properties
Original file line number Diff line number Diff line change
@@ -0,0 +1,25 @@
# OKafka Consumer example properties

#Properties to connect to Oracle Database
#Option 1: Connect to Oracle database using plaintext
bootstrap.servers=<server address:server port>
oracle.service.name=<oracle database service>
oracle.net.tns_admin=<location of ojdbc.properties file>


#Option 2: Connect to Oracle Database deployed in Oracle Autonomous Cloud using Wallet
#security.protocol=SSL
#oracle.net.tns_admin=<location of Oracle Wallet, tnanames.ora and ojdbc.properties file>
#tns.alias=<tns alias>

# Application specific OKafka consumer properties
topic.name=<Oracle Database TxEventQ Topic, use uppercase>
group.id=<Oracle Database TxEventQ Subscriber>

enable.auto.commit=true
max.poll.records=1000
default.api.timeout.ms=180000

key.deserializer=org.apache.kafka.common.serialization.StringDeserializer
value.deserializer=org.apache.kafka.common.serialization.StringDeserializer

2 changes: 1 addition & 1 deletion examples/producer/src/main/resources/config.properties
Original file line number Diff line number Diff line change
Expand Up @@ -13,7 +13,7 @@ oracle.net.tns_admin=<location of ojdbc.properties file>
#tns.alias=<tns alias>

#Appliction specific OKafka Producer properties
topic.name=<Oracle Database TxEventQ Topic>
topic.name=<Oracle Database TxEventQ Topic, use uppercase>

batch.size=200
linger.ms=100
Expand Down