LIBRSS(3)

NAME
librss - Provide Receive-side scaling awareness to userland applications

LIBRARY
librss

SYNOPSIS
#include <sys/param.h>
#include <sys/cpuset.h>
#include <librss.h>
struct rss_config *
rss_config_get(void);
void
rss_config_free(struct rss_config *cfg);
int
rss_config_get_bucket_count(struct rss_config *cfg);
int
rss_get_bucket_cpuset(struct rss_config *rc, rss_bucket_type_t btype, int bucket, cpuset_t *cs);
int
rss_set_bucket_rebalance_cb(rss_bucket_rebalance_cb_t *cb, void *cbdata);
int
rss_sock_set_recvrss(int fd, int af, int val);

DESCRIPTION
The librss library and the functions it provides are used for both fetching the system RSS configuration and interacting with RSS aware sockets.

Applications will typically call rss_config_get() to fetch the current RSS configuration from the system and perform initial setup. This typically involves spawning worker threads, one per RSS bucket, and optionally binding them to the per-bucket CPU set.

The rss_config struct is defined as:

    struct rss_config {
    	int rss_ncpus;
    	int rss_nbuckets;
    	int rss_basecpu;
    	int *rss_bucket_map;
    };

Applications will typically use the rss_config_get_bucket_count() function to fetch the number of RSS buckets, create one thread per RSS bucket for RSS aware work, then one RSS aware socket to receive UDP datagrams or TCP connections in each particular RSS bucket / thread.

The rss_get_bucket_cpuset() function sets the given cpuset up for the given RSS bucket and behaviour. Typically applications will wish to just query for RSS_BUCKET_TYPE_KERNEL_ALL unless they wish to potentially setup different worker threads for transmit and receive.

The rss_bucket_type_t enum is defined as:

    typedef enum {
            RSS_BUCKET_TYPE_NONE = 0,
            RSS_BUCKET_TYPE_KERNEL_ALL = 1,
            RSS_BUCKET_TYPE_KERNEL_TX = 2,
            RSS_BUCKET_TYPE_KERNEL_RX = 3,
            RSS_BUCKET_TYPE_MAX = 3,
    } rss_bucket_type_t;

The rebalance callback rss_bucket_rebalance_cb_t is defined as:

    typedef void rss_bucket_rebalance_cb_t(void *arg);

The rss_set_bucket_rebalance_cb() function sets an optional callback that will be called if the kernel rebalances RSS buckets. This is intended as a future expansion to rebalance buckets rather than reprogram the RSS key, so typically the only work to be performed is to rebind worker threads to an updated cpuset.

Once RSS setup is completed, rss_config_free() is called to free the RSS configuration structure.

If val is set to 1, the socket can be placed in an RSS bucket and will only accept datagrams (for UDP) or connections (for TCP) that are received for that RSS bucket. If set to 0, the socket is placed in the default PCB and will see datagrams/connections that are not initially consumed by a PCB aware socket.

The rss_sock_set_recvrss() function enables or disables receiving RSS related information as socket options in recvmsg(2) calls.

When enabled, UDP datagrams will have a message with the IP_RECVFLOWID option indicating the 32-bit receive flowid as a uint32_t, and the IP_RECVRSSBUCKETID option indicating the 32 bit RSS bucket id as a uint32_t.

ERRORS
The functions return either <0 or NULL as appropriate upon error.

HISTORY
The librss.3 library first appeared in FreeBSD 11.0.

AUTHORS
Adrian Chadd <adrian@FreeBSD.org>

BUGS
There is currently no kernel mechanism to rebalance the RSS bucket to CPU mapping, and so the callback mechanism is a no-op.