This document describes the design of OWSL in order to help in development of a transport layer implementation.
OWSL components
OWSL has a BSD-like socket API.
The difference with the BSD socket is the management of socket type: many implementations can exist and owsl_socket_type
has an API to register the prefered implementations. On socket creation OWSL choose an implementation among the registered ones which match with the type parameters.
OWSL extends the BSD socket API with features like callback, socket name...
The component owsl_address
is a tool to manage addresses. With this component, an application can use IPv6 and IPv4 transparently.
owsl_socket
contains functions for handle management, socket structure creation and socket informations.
owsl_asynchronous
contains functions for callback and select()
mechanism.
owsl_monitor
has a dynamically configurable thread to manage system sockets by using select()
. It can be controlled by the implementation and send events to a callback in the implementation.
A transport protocol added in OWSL is called implementation. It could also be called a plugin.
owqueue
is a circular FIFO with transaction, callback and packet features. It makes a lot of work in OWSL like handling blocking and non blocking mode.
owlist
is a thread-safe linked list.
pthread
is used for thread, mutexes, lock, etc...
To illustrate the architecture, the following diagramm shows a reception through all the layers.
The monitor and the queue layers could be draw at the same level because they are totally independents. The implementation is located between the queue/monitor layer at the bottom and the asynchronous/user API layer at the upper. To simplify, the API is not draw. That is why the user application is directly connected to the implementation but it is not the truth.
A circle represent a waiting loop in a thread. It is woken up by an event represented by a zigzag arrow. A trapezoid is a blocking function that is also woken up by an event.
architecture of an OWSL receive
When a packet is received by the OS, the monitor which was blocking on select()
is woken up. It calls the callback function of the implementation and pass the event in parameter.
The implementation receives the packet from the system socket and writes the packet in its input queue. If it is full, the queue sends the event to the implementation by calling a dedicated callback. To prevent the monitor from signalling another incoming packet that cannot be stored, the implementation disables the monitoring of incoming packet on this socket.
After having received the packet, the implementation notifies the asynchronous layer with the read event. It has two effects: the event is sent to the callback launcher and a signal is sent to wake up the blocking asynchronous functions like owsl_select()
. This way, all the asynchronous mechanism are trigerred.
The user may read the packet by a call to owsl_recv()
in its callback or after a select or elsewhere. If it is in blocking mode, and if the input queue is empty then it blocks until a packet is write in the queue. At this moment, the read function of the queue sends a signal to the write function to unlock it. In another case, if the queue was full before the read, then the queue sends a signal to the implementation. So the queue callback can reenable the monitoring that was disabled when the queue became full.
Let's see what is needed to implement a transport protocol in OWSL.
It is important to keep in mind that a maximum of things are managed in the core to keep the protocol implementation as simple as possible.
An implementation (or plugin) is also called a socket type. In fact, each implementation must define one or many socket type. In example, UDP implementation defines two types: UDP/IPv4 and UDP/IPv6. These types are defined in the enumeration OWSLSocketType
and in instances of the OWSLSocketTypeInfo structure.
struct OWSLSocketTypeInfo
{
OWSLSocketType type ; /** unique identifier of socket type */
OWSLAddressFamily address_family ; /** parameter of socket type */
OWSLSocketMode mode ; /** parameter of socket type */
OWSLCiphering ciphering ; /** parameter of socket type */
int (* global_parameter_set) (const char * name, const void * value) ;
int (* is_readable) (OWSLSocketInfo * socket) ;
int (* is_writable) (OWSLSocketInfo * socket) ;
int (* has_error) (OWSLSocketInfo * socket) ;
int (* blocking_mode_set) (OWSLSocketInfo * socket, OWSLBlockingMode mode) ;
int (* parameter_set) (OWSLSocketInfo * socket, const char * name, const void * value) ;
struct sockaddr * (* remote_address_get) (OWSLSocketInfo * socket) ;
OWQueueCallback on_queue_event ;
OWSLSocketInfo * (* socket) (OWSLSocketType type) ;
OWSLSocketInfo * (* accept) (OWSLSocketInfo * socket, struct sockaddr * address, int * address_length) ;
int (* close) (OWSLSocketInfo * socket) ;
int (* bind) (OWSLSocketInfo * socket, const struct sockaddr * address, int address_length) ;
int (* connect) (OWSLSocketInfo * socket, const struct sockaddr * address, int address_length) ;
int (* listen) (OWSLSocketInfo * socket, int pending_max) ;
int (* send) (OWSLSocketInfo * socket, const void * buffer, int length, int flags) ;
int (* recv) (OWSLSocketInfo * socket, void * buffer, int size, int flags) ;
int (* sendto) (OWSLSocketInfo * socket, const void * buffer, int length, int flags, const struct sockaddr * address, int address_length) ;
int (* recvfrom) (OWSLSocketInfo * socket, void * buffer, int size, int flags, struct sockaddr * address, int * address_length) ;
}
address_family
, mode
and ciphering
classify the socket type and are used by owsl_socket()
to retrieve the right socket type.
global_parameter_set
should be defined if the protocol needs to use global parameters.
is_readable
, is_writable
and has_error
should be defined if the functions owsl_is_readable
, owsl_is_writable
and owsl_has_error
(in owsl_socket.c) are not sufficients for the implementation.
blocking_mode_set
should be defined if the settings of the blocking mode of the queues are not sufficients for the implementation.
parameter_set
should be defined if the protocol needs to use special parameters which are different for each socket.
remote_address_get
should be defined if the protocol has a connected mode.
queue_callback
should be defined if the queues are used.
The BSD socket functions should be defined if they are relevant for the implementation. If it is NULL, the top level function will return an error.
All the OWSL sockets have the same base: OWSLSocketInfo
.
struct OWSLSocketInfo
{
OWSLSocket socket ; /** handle for higher level layer */
struct OWSLSocketTypeInfo * type_info ; /** type of the socket */
OWSLBlockingMode blocking_mode ; /** synchonous or asynchronous */
OWQueue * in_queue ; /** queue that contains data received from network */
OWQueue * out_queue ; /** queue that contains data to send over network */
pthread_mutex_t listening_mutex ; /** listen is exclusive with connect, send, recv... */
int listening ; /** 1 if listen was called, -1 if listen cannot be called */
int connected ; /** 1 when really connected after connect or accept */
int error ; /** should be 0 */
OWSLCallback callback_function ; /** called at each event */
void * callback_user_data ; /** parameter for callback function */
OWSLAddress bound_address ; /** local bound address */
char * name ; /** name of the socket, used for debugging purposes */
}
The first field is a simple identifier for the socket. It is used to call the functions of the API.
type_info
is the link to the socket type structure.
blocking_mode
is used and managed by the core.
in_queue
and out_queue
can be very important in the implementation. It is not mandatory to use it but could be a great help... It can store data, packet informations and can handle blocking/non blocking mode and atomic transaction.
listening_mutex
is used by the core to protect listening
.
listening
is set by the core. It can be tested to know if listen()
was called for this socket. It is used by the asynchronous layer.
connected
must be set by the implementation when the socket is really connected. It is used by the asynchronous layer.
error
is set by the core when an error is notified by the implementation. It is used by the asynchronous layer.
callback_function
and callback_user_data
are managed by the core. It is used by the asynchronous layer.
address
is set by the core after a sucessful bind.
name
is managed by the core.
To keep a common socket structure between different implementations and allow specific parameters, the socket structure must be inherited by the specific socket structure of the implementation. An example for TCP:
typedef struct OWSLSocketInfo_TCP
{
OWSLSocketInfo base ; /* must be the first field for inherited behaviour */
OWSLSystemSocket system_socket ;
OWSLAddress remote_address ;
int remote_address_length ;
} OWSLSocketInfo_TCP ;
This structure can be casted in OWSLSocketInfo
because the common part is in first position.
The handle is totally managed by the core. No need to care about it.
A socket structure must be allocated with the function owsl_socket_info_new()
. This function takes parameters for the socket type, the total size of the specific structure and the queues parameters (if used). Example for TCP:
OWSLSocketInfo * socket_info ;
socket_info = owsl_socket_info_new
(
type, /* socket type */
sizeof (OWSLSocketInfo_TCP), /* size of socket structure */
OWSL_TCP_PACKET_AVG_SIZE * OWSL_QUEUE_PACKET_MAX, /* queues usable size */
OWQUEUE_NO_PACKET, /* queues mode */
OWSL_QUEUE_PACKET_MAX, /* queues packet maximum */
0, /* queues packet info size */
1, /* create input queue */
1 /* create output queue */
) ;
if (socket_info != NULL)
{
OWSLSocketInfo_TCP * socket_tcp = (OWSLSocketInfo_TCP *) socket_info ;
/* TCP specific stuff */
/* ... */
}
Only the specific parameters of the implementation need to be initialized after allocating the structure.
The socket must be in blocking mode by default (including for connect
and accept
functions).
To make a new implementation in OWSL, the core must be modified at three places:
OWSLSocketType
enumeration (in owsl.h).OWSLSocketTypeInfo
instance and send it to the core by the function owsl_socket_type_initialize
.owsl_socket_type_initialize_all
.As it is done for the initialization, a cleaning function can be called in owsl_socket_type_terminate_all if needed.
OWSL can be used in blocking or non blocking mode. To make the non blocking mode usable, an asynchronous management must be done. It can be handle by many ways:
owsl_select()
as in the BSD APIowsl_poll()
as in the BSD API (not yet implemented)The only thing to do in the implementation is to notify the core at each event:
OWSL_EVENT_READ
if there is a new packet or a new incoming connectionOWSL_EVENT_WRITE
if a packet can be sent and that it was not possible beforeOWSL_EVENT_ERROR
if the socket is no more usableThe function owsl_notify()
calls owsl_signal()
to unlock waiting threads and owsl_callback()
to send the event in the callback function which is asynchronously called in a dedicated thread.
The monitor should be used if the implementation uses a BSD socket. It helps to manage the socket by providing a thread which is blocking on select()
for the registered sockets on the registered events.
owsl_monitor_socket_add()
and owsl_monitor_socket_remove()
must not be called in the monitor callback function.
By adding or removing events to monitor, the implementation can decide if it is ready to receive an event. For example, if a packet is received in the BSD socket and the OWSL input queue is full, then monitor send many read events until the packet is removed from the BSD socket. To prevent from a polling which consumes the CPU, the implementation should remove the read event from the monitor until its input queue is no more full.
To use the monitor, the BSD socket must be registered in. It automatically registers the socket for the error event. Read and write event must be "manually" added. The monitor will send events by calling the registered callback function. The last parameter of the callback is as usual an user data pointer ; it should be set to the socket structure.
It is possible to register an event as a special one. If the special flag OWSL_MONITOR_ONCE
is added, the events that was previously monitored for this socket are ignored until this special event arrives. It is signalled in the callback with the special flag. Then it is removed from the once events of the socket. If there is no other once event, the normal events are monitored.
A lot of work can be done by the owsl_queue. That is why its usage is highly recommended.
Input and output queues are initialized (if enabled) by the core when creating the socket. The queue callback of the implementation is automatically registered for each queue. The callback user data is the socket structure.
System socket (also called BSD socket) can be managed with unified type and functions for Windows and POSIX systems:
OWSLSystemSocket
is the unified typeowsl_system_socket_blocking_mode_set
is the unified function to set the blocking modeOWSL_SYSTEM_SOCKET_CLOSE
is the unified close function (defined as a macro)OW_MEMCPY
can be useful to implement accept and recvfrom. It makes a safe copy of data and copy data length.
OW_GET_ERRNO
and OW_SET_ERRNO
can be used to handle error codes on POSIX systems and Windows ones.