-
Notifications
You must be signed in to change notification settings - Fork 132
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Protocol specification. #28
Comments
+1 These give you a good sense of the structure of the messages, but less so whats a valid response to a particular communication to the broker. |
Hi @kafkiansky, thanks for your interest in the product! What @willhoy has pointed out are basically the two files which capture the wire protocol containing low level structs and control messages (encoded requests/responses). However, these files don't explain the order of exchange of these structs/messages/requests/responses, as well as any contract between client and broker around that exchange. We have not documented this yet. However, we have a sample producer and consumer implementation which does not use client library. Instead, it simply uses socket APIs to connect/read/write to the broker and sends/receives messages to/from the broker. While this is not a formal spec, it should be enough to get you started to have a rudimentary working prototype. I would also like to add that in order to provide a good experience, the C++ and Java client libraries are quite stateful. Asychronous nature of the APIs also adds complexity in the implementation. Generally speaking, implementing a batteries-included library is a non-trivial effort. However, we are happy to help you out if you decide to implement one. Out of curiosity, what language do you have in mind for the client library? |
Hi, @quarter-note. Thanks for the explanation. I would like to implement the libraries in rust, go and php. I will try to use the information from those files you gave, or will watch tcpdump as a last resort. |
Ok great, I will try to write a high level overview of the message flow and some other relevant details. |
Thank you so much for your help. |
Hi @kafkiansky Just wanted to mention that we have started working on documenting the protocol. It's WIP but it can be found here. Hoping to wrap it up in the coming weeks. |
Hi, @quarter-note. Great, thank you. I've already started reverse-engineering your protocol using a Java library, but it would be much easier with a formal specification. |
Hi @kafkiansky I have added some more details in the document. It's still WIP, but you may find newly added info useful. |
Hi @quarter-note. Yes, I am following the changes and going to continue development this weekend. Thank you for your work! |
@quarter-note, I didn't see any information about what order you write the bytes in: little endian or big endian? |
I would be very surprised if the bytes were not written in 'Network byte order' using the equivalent of |
So it's BigEndian. Thanks for the hint, @willhoy. |
Yes, it's big endian (or network byte order). Some additional notes -- in Java, we have two wrapper classes In C++, we use a |
Added some info in the doc. |
Hi, @quarter-note. While developing an SDK, I'm trying to run an OpenQueue request, but I'm getting an error Perhaps the bmqbrkr logs will be useful: bmqbrkr_1 | 19AUG2023_18:08:49.300 (139933851313728) INFO mqba_domainresolver.cpp:190 Error reading the domain config file [domain: 'myapp', rc: -1, output:
bmqbrkr_1 | Domain file '/etc/local/bmq/domains/myapp.json' doesn't exist
bmqbrkr_1 | 19AUG2023_18:08:49.300 (139933851313728) WARN mqbblp_queuesessionmanager.cpp:114 #CLIENT_OPENQUEUE_FAILURE local:[email protected]:55432: Error while qualifying domain: [ category = E_UNKNOWN code = -1 message = "Domain file '/etc/local/bmq/domains/myapp.json' doesn't exist" ], request: [ rId = 1 choice = [ openQueue = [ handleParameters = [ uri = "bmq://myapp/queue" qId = 0 subIdInfo = NULL flags = 12 readCount = 0 writeCount = 1 adminCount = 0 ] ] ] ]
bmqbrkr_1 | 19AUG2023_18:08:49.300 (139933851313728) INFO mqba_clientsession.cpp:349 local:[email protected]:55432: Sending openQueue failure response: [ rId = 1 choice = [ status = [ category = E_UNKNOWN code = -1 message = "Domain file '/etc/local/bmq/domains/myapp.json' doesn't exist" ] ] ]
bmqbrkr_1 | 19AUG2023_18:08:49.303 (139933074130496) INFO mqbnet_tcpsessionfactory.cpp:734 TCPSessionFactory 'TCPInterface': OnClose channel [session: 'local:[email protected]:55432', channel: '172.19.0.1:55432#0x7f44c001c768', 0 active channels, status: [ Category = SUCCESS ]] |
Hi @kafkiansky You try to use domain From the path This folder is connected to the container as a volume here:
So to fix this problem, you can either:
|
Also some info about domains: |
@678098, thank you for the explanation. |
@kafkiansky I have raised PR #95. Feel free to take a look and comment. |
Hello, @quarter-note. In the process of developing sdk on go, I am focusing on java sdk. There I noticed different packing of properties depending on the flag https://github.com/bloomberg/blazingmq-sdk-java/blob/main/bmq-sdk/src/main/java/com/bloomberg/bmq/impl/infr/proto/MessagePropertiesImpl.java#L416. What does this mean? Does this need to be supported or can only "newStyleProperties" be used? |
Hello, @678098. Maybe you know the answer to the question above? |
Hi @kafkiansky! Only the new style message properties are needed. They are called "message properties V2" or "extended message properties". Some details about them are in this comment: #73 (comment) |
Thank you, @678098. |
Is there an existing proposal for this?
Is your feature request related to a problem?
I want to implement a client to your broker.
Describe the solution you'd like
Will there be a protocol specification added?
Alternatives you considered
No response
The text was updated successfully, but these errors were encountered: