Migration to bindings v4
-
Version 1, Monday 4 May 2020
- Initial document
-
Version 2, Tuesday 5 May 2020
- Add buffer creation/scanning to list of needed evolutions
- Add reference to mime types
-
Version 3, Thursday 20 April 2023
- Improved migration guide
At this state of the document, anything here is PROPOSAL and subject to discussions
Introduction
The already existing micro-service framework afb-binding was improved during year 2020 in order to solve some existing problems reveled by usage. This work introduced the fourth version of the afb-binding API also called binding v4.
The main reason for shifting from v3 to v4 is the poor performances measured when using JSON encoding that is mandatory in v3.
These poor performances can be fighten by using an other library for JSON, a library that would enforce the serialisation instead of the internal data manipulation. A quick study showed that the main cause would remain and that JSON encoding is a flaw, not the library used.
A change of serialisation format, for a more efficient one, like CBOR, could also lead to the same situation.
Solving any of the above issues with the API version 3 of bindings is not wanted because after that the programming model would not be clear.
During 18th week of 2020, we took the decision to create a new version of API for bindings, the version 4.
The constraints were:
-
No more link to JSON serialisation or internal representation but be open to any internal representation or serialisation.
-
It is accepted that existing V3/JSON client would not be fully able to interact with new bindings. It is also accepted that clients compatible with V4 might not communicate with bindings V4 that it don’t know because it has no way to exploit an unknown formatting data.
-
JSON data currently used are kept for compatibility and for its good convenience but are seen as a JSON string by default, not as objects of libjson-c. The transformation to object of json-c library would be still possible but on request.
Overview of operations
Clients and servers manipulates 3 kinds of items: events, requests and replies. The table below summarizes operations performed.
| server | client | |
|---|---|---|
| event | create | scan |
| request | scan | create |
| reply | create | scan |
The request is adressed to a destination designated by the pair API and VERB.
The reply is adressed to the caller and includes from the beginning a status, indicating whether it is a success or an error and a companion string named info. Having a synthetic OK versus KO status companion of the reply is probably needed but because the new version of API changes how data are handled, it is a valid question to ask if the string error and info are to be kept as is, as aside of the answer, or if it should be managed by the implementers as part of the transmited data.
At the end, the binder that is in charge of transmitting data could do it using only 2 operations easy to abstract: serialization and deserialization of the data.
Naive example of operation’s flow:
- the client creates a request
- the client serializes the request to a flow
- the client transmits the flow to the binder
- the binder receives the flow
- the binder transmits the flow to the binding
- the binding deserializes the flow to a request
- the binding treats the request
- the binding creates the reply
- the binding serializes the reply to a flow
- the binding transmits the flow to the binder
- the client deserializes the flow to a reply
That flow is already mostly in place except that the binder, not the binding, serializes and deserializes the flow for the most common type of data, the predefined data types.
For the application specific data types, the binder invoque application code to serialize and deserialize the data flow on need.
Data model for bindings V4
Type of data
With previous versions, the type of the data was implicitly JSON plus some nearby items linked to the first binding interface HTML. So key/values were still existing.
Now the type of data must be given explicitly.
How? Strings? UUID? Integers? Mime types?
Creation of data
In all cases, the creation must be started by indicating the type of the data created.
To achieve performance by avoiding unneeded memory copy and to avoid issues resulting of blocking I/Os, a mix of technics has to be used:
-
for fresh buffer to create, the binder should be able to allocate it for the client and return its address
-
for static buffers (like constant strings), or already allocated buffers, it should handle the address, the length and a release callback
When pushed to the binder as reply or as event, the data is released by the binder.
Scanning the data
The received data is always read only. It has a type.
The scanner request access to read only buffer. The binder can deliver the buffer by pieces.
Question about streaming
In some cases the data to transmit is big (mega is already big, but what about giga?) or is never terminating (audio/video streams).
Should the framework handle it? If no, what is the limit?
The current specification expects that when data is really big, the application must slice it. There is no enforced limit, but developpers and architects in charge of defining working system must handle the question because the version 4 of afb-binding is not providing a streaming API.
Evolutions of the API
The shift to binding V4 will bring new things. Some are enforced by the need. Some could be discussed.
Needed:
- registration / management of types of payload
- versionning?
- buffer creation and scanning
Optional:
- new interface for managing sessions
Comparisons between V3 and V4
Comparison of afb_binding_t
| v3 | V4 | comment |
|---|---|---|
| api | api | unchanged |
| specification | specification | unchanged |
| info | info | unchanged |
| verbs | verbs | signature changed |
| preinit | - | removed |
| init | - | removed |
| - | mainctl | main callback |
| onevent | onevent | removed |
| userdata | userdata | unchanged |
| provide_class | provide_class | unchanged |
| require_class | require_class | unchanged |
| require_api | require_api | unchanged |
| noconcurrency | noconcurrency | unchanged |
Comparison fo afb_verb_t
| v3 | V4 | comment |
|---|---|---|
| verb | verb | unchanged |
| callback | callback | signature changed |
| auth | auth | unchanged |
| info | info | unchanged |
| vcbdata | vcbdata | unchanged |
| session | session | unchanged |
| glob | glob | unchanged |
Comparison of afb_auth_t
Are the same, that’s all.
Comparison of SESSION
| v3 | V4 | comment |
|---|---|---|
| AFB_SESSION_NONE | AFB_SESSION_NONE | unchanged |
| AFB_SESSION_CLOSE | AFB_SESSION_CLOSE | unchanged |
| AFB_SESSION_CHECK | AFB_SESSION_CHECK | unchanged |
| AFB_SESSION_LOA_0 | AFB_SESSION_LOA_0 | unchanged |
| AFB_SESSION_LOA_1 | AFB_SESSION_LOA_1 | unchanged |
| AFB_SESSION_LOA_2 | AFB_SESSION_LOA_2 | unchanged |
| AFB_SESSION_LOA_3 | AFB_SESSION_LOA_3 | unchanged |
| AFB_SESSION_RENEW | - | removed, ignored, act fact that tokens are generated outside |
| AFB_SESSION_REFRESH | - | removed, ignored, act fact that tokens are generated outside |
Comparison of afb_api_t
| v3 | V4 | comment |
|---|---|---|
| afb_api_name | afb_api_name | kept, but not more a macro |
| afb_api_get_userdata | afb_api_get_userdata | kept, but not more a macro |
| afb_api_set_userdata | afb_api_set_userdata | kept, but not more a macro |
| afb_api_logmask | afb_api_logmask | kept, but not more a macro |
| afb_api_wants_log_level | afb_api_wants_log_level | unchanged |
| afb_api_vverbose | afb_api_vverbose | unchanged |
| afb_api_verbose | afb_api_verbose | unchanged |
| afb_api_get_event_loop | - | removed |
| afb_api_get_user_bus | - | removed |
| afb_api_get_system_bus | - | removed |
| afb_api_rootdir_get_fd | - | removed |
| afb_api_rootdir_open_locale | - | removed |
| afb_api_queue_job | - | replaced by afb_job_queue |
| afb_api_require_api | afb_api_require_api | unchanged |
| afb_api_broadcast_event | afb_api_broadcast_event | signature changed |
| afb_api_make_event | - | replaced by afb_api_new_event |
| - | afb_api_new_event | new |
| afb_api_call | afb_api_call | signature changed |
| afb_api_call_sync | afb_api_call_sync | signature changed |
| afb_api_call_legacy | - | removed |
| afb_api_call_sync_legacy | - | removed |
| afb_api_new_api | - | replaced by afb_create_api |
| afb_api_delete_api | - | replaced by afb_api_delete |
| - | afb_api_delete | new |
| afb_api_set_verbs_v2 | - | removed |
| afb_api_set_verbs_v3 | - | removed |
| - | afb_api_set_verbs | new |
| afb_api_add_verb | afb_api_add_verb | signature changed |
| afb_api_del_verb | afb_api_del_verb | unchanged |
| afb_api_on_event | afb_api_on_event | signature changed |
| afb_api_on_init | afb_api_on_init | unchanged |
| afb_api_seal | afb_api_seal | unchanged |
| afb_api_add_alias | afb_api_add_alias | unchanged |
| afb_api_event_handler_add | afb_api_event_handler_add | signature changed |
| afb_api_event_handler_del | afb_api_event_handler_del | unchanged |
| afb_api_require_class | afb_api_require_class | unchanged |
| afb_api_provide_class | afb_api_provide_class | unchanged |
| afb_api_settings | afb_api_settings | signature changed |
Comparison of afb_event_t
| v3 | V4 | comment |
|---|---|---|
| afb_event_is_valid | afb_event_is_valid | unchanged |
| afb_event_unref | afb_event_unref | unchanged |
| afb_event_addref | afb_event_addref | unchanged |
| afb_event_name | afb_event_name | unchanged |
| afb_event_broadcast | afb_event_broadcast | signature changed |
| afb_event_push | afb_event_push | signature changed |
Comparison of afb_req_t
| v3 | V4 | comment |
|---|---|---|
| afb_req_is_valid | afb_req_is_valid | unchanged |
| afb_req_get_api | afb_req_get_api | unchanged |
| afb_req_get_vcbdata | afb_req_get_vcbdata | unchanged |
| afb_req_get_called_api | afb_req_get_called_api | unchanged |
| afb_req_get_called_verb | afb_req_get_called_verb | unchanged |
| afb_req_wants_lo_level | afb_req_wants_log_level | unchanged |
| afb_req_get | - | removed |
| afb_req_value | - | removed |
| afb_req_path | - | removed |
| afb_req_json | - | removed but migration facility |
| afb_req_reply | afb_req_reply | signature changed |
| afb_req_reply_f | - | removed |
| afb_req_reply_v | - | removed |
| afb_req_context_get | afb_req_context_get | unchanged |
| afb_req_context_set | afb_req_context_set | unchanged |
| afb_req_context | afb_req_context | unchanged |
| afb_req_context_make | afb_req_context_make | unchanged |
| afb_req_context_clear | afb_req_context_clear | unchanged |
| afb_req_addref | afb_req_addref | unchanged |
| afb_req_unref | afb_req_unref | unchanged |
| afb_req_session_close | afb_req_session_close | unchanged |
| afb_req_session_set_LOA | afb_req_session_set_LOA | unchanged |
| afb_req_subscribe | afb_req_subscribe | unchanged |
| afb_req_unsubscribe | afb_req_unsubscribe | unchanged |
| afb_req_subcall | afb_req_subcall | unchanged |
| afb_req_subcall_legacy | - | removed |
| afb_req_subcall_req | - | removed |
| afb_req_subcall_sync_legacy | - | removed |
| afb_req_subcall_sync | afb_req_subcall_sync | unchanged |
| afb_req_verbose | afb_req_verbose | unchanged |
| afb_req_has_permission | - | removed |
| afb_req_check_permission | afb_req_check_permission | unchanged |
| afb_req_get_application_id | - | removed |
| afb_req_get_uid | - | removed |
| afb_req_get_client_info | afb_req_get_client_info | signature changed |
| afb_req_success | - | removed |
| afb_req_success_f | - | removed |
| afb_req_success_v | - | removed |
| afb_req_fail | - | removed |
| afb_req_fail_f | - | removed |
| afb_req_fail_v | - | removed |
| afb_stored_req | - | removed |
| afb_req_store | - | removed |
| afb_req_unstore | - | removed |
new features
| V4 | comment |
|---|---|
| afb_create_api | replaces afb_api_new_api |
| afb_job_queue | replaces afb_api_queue_job and improved |
| afb_job_abort | new |
| afb_req_get_userdata | new |
| afb_req_set_userdata | new |
Standardisation of error codes
Replies of requests in V4 include an integer status. That integer status is meant to either be an error code or a short answer. By convention negative values are for errors and positives or null values are for short answers.
The macros AFB_IS_ERRNO, AFB_USER_ERRNO, AFB_IS_BINDER_ERRNO provide user’s method to check for reply status:
| value’s range | AFB_IS_ERRNO(value) | AFB_IS_USER_ERRNO(value) | AFB_IS_BINDER_ERRNO(value) | meaning |
|---|---|---|---|---|
| value >= 0 | false | false | false | application status |
| -1 >= value >= -999 | true | false | true | predefined error |
| value >= -1000 | true | true | false | application error |
It is allowed by the convention that application use a predefined error status when applicable.
The predefined error code are:
| error name | HTTP | usable | Meaning |
|---|---|---|---|
| AFB_ERRNO_INTERNAL_ERROR | 500 (INTERNAL SERVER ERROR) | yes | Any unclassified internal error |
| AFB_ERRNO_OUT_OF_MEMORY | 500 (INTERNAL SERVER ERROR) | yes | Specific to memory depletion |
| AFB_ERRNO_UNKNOWN_API | 404 (NOT FOUND) | Invoked API does not exist | |
| AFB_ERRNO_UNKNOWN_VERB | 404 (NOT FOUND) | Invoked verb does not exit | |
| AFB_ERRNO_NOT_AVAILABLE | 501 (NOT IMPLEMENTED) | yes | The service is not available |
| AFB_ERRNO_UNAUTHORIZED | 401 (UNAUTHORIZED) | yes | The client miss authorization |
| AFB_ERRNO_INVALID_TOKEN | 401 (UNAUTHORIZED) | The client has an invalid token | |
| AFB_ERRNO_FORBIDDEN | 402 (FORBIDDEN) | yes | The client has not the permission to perform the action |
| AFB_ERRNO_INSUFFICIENT_SCOPE | 402 (FORBIDDEN) | yes | The scope associated to the client’s token is insuficient |
| AFB_ERRNO_BAD_API_STATE | 402 (FORBIDDEN) | yes | The called API is not ready |
| AFB_ERRNO_NO_REPLY | The service did not reply | ||
| AFB_ERRNO_INVALID_REQUEST | yes | Some argument of the request are invalid | |
| AFB_ERRNO_NO_ITEM | yes | An item is missing, depend on the context | |
| AFB_ERRNO_BAD_STATE | yes | The current state can accept the request | |
| AFB_ERRNO_DISCONNECTED | Connection to the service is broken |
The column usable means that the error can be used by applications.