Skip to content

Understanding sharding

Sharding

In seyfert the sharding approach is to give the full benefit of scaling while keeping the same structure in your project.

Why sharding?

JavaScript runtimes are single-threaded, which means that without sharding, all processing would happen in a single thread. This creates a processing limit where all the load is evaluated together. Seyfert handles sharding internally in the Client instance to help distribute this load across multiple processes or threads (although it is still an imperfect technique).

Managing shards

The base of the Worker is to allow to execute code in parallel in different parts of the CPU, either in threads or different processes. In terms of discord, this means the ability to connect several shards by spreading the load on each Worker, for seyfert this is just changing the mode property in the WorkerManager to decide the execution mode between threads to spawn clients on processor threads or cluster to spawn clients on different processes of the runtime.

import {
class WorkerManager
WorkerManager
} from 'seyfert';
const
const manager: WorkerManager
manager
= new
new WorkerManager(options: Omit<PickPartial<WorkerManagerOptions, "token" | "intents" | "info" | "handlePayload" | "handleWorkerMessage">, "resharding"> & {
resharding?: PickPartial<NonNullable<WorkerManagerOptions["resharding"]>, "getInfo">;
}): WorkerManager
WorkerManager
({
mode: "threads" | "clusters" | "custom"
mode
: "threads",
// ./src/client.ts for bun and deno (?
path: string
path
: "./dist/client.js",
// you can override a lot of options, like number of workers, shards per worker...
});
const manager: WorkerManager
manager
.
WorkerManager.start(): Promise<void>
start
();

Too simple? Seyfert takes care of all the logic so your project shouldn’t change much just by switching to a WorkerSharding.

Cache

Unlike traditional Discord libraries, Seyfert offers unified cache management across all shards. The cache can be centralized in the main process (the WorkerManager executor), ensuring consistent data access throughout your application.

To implement centralized caching, use the WorkerAdapter:

import {
class WorkerClient<Ready extends boolean = boolean>
WorkerClient
,
class WorkerAdapter
WorkerAdapter
} from 'seyfert';
const
const client: WorkerClient<boolean>
client
= new
new WorkerClient<boolean>(options?: WorkerClientOptions): WorkerClient<boolean>
WorkerClient
();
const client: WorkerClient<boolean>
client
.
WorkerClient<boolean>.setServices(rest: ServicesOptions): void
setServices
({
ServicesOptions.cache?: {
adapter?: Adapter;
disabledCache?: boolean | DisabledCache | ((cacheType: keyof DisabledCache) => boolean);
}
cache
: {
adapter?: Adapter
adapter
: new
new WorkerAdapter(workerData: WorkerData): WorkerAdapter
WorkerAdapter
(
const client: WorkerClient<boolean>
client
.
WorkerClient<boolean>.workerData: WorkerData
workerData
)
}
});
await
const client: WorkerClient<boolean>
client
.
WorkerClient<boolean>.start(options?: Omit<DeepPartial<StartOptions>, "httpConnection" | "token" | "connection">): Promise<void>
start
();

Talking to other workers

If for some reason (I did not find any for the example), you want a specific worker to execute an action that another one received, you can simply ask it.

const client: WorkerClient<boolean>
client
.
WorkerClient<boolean>.tellWorker<void, {
workerId: number;
}>(workerId: number, func: (_: WorkerClient<boolean>, vars: {
workerId: number;
}) => void, vars: {
workerId: number;
}): Promise<void>
tellWorker
(
1,
(
worker: WorkerClient<boolean>
worker
,
vars: {
workerId: number;
}
vars
) =>
var console: Console

The console module provides a simple debugging console that is similar to the JavaScript console mechanism provided by web browsers.

The module exports two specific components:

  • A Console class with methods such as console.log(), console.error() and console.warn() that can be used to write to any Node.js stream.
  • A global console instance configured to write to process.stdout and process.stderr. The global console can be used without importing the node:console module.

Warning: The global console object's methods are neither consistently synchronous like the browser APIs they resemble, nor are they consistently asynchronous like all other Node.js streams. See the note on process I/O for more information.

Example using the global console:

console.log('hello world');
// Prints: hello world, to stdout
console.log('hello %s', 'world');
// Prints: hello world, to stdout
console.error(new Error('Whoops, something bad happened'));
// Prints error message and stack trace to stderr:
// Error: Whoops, something bad happened
// at [eval]:5:15
// at Script.runInThisContext (node:vm:132:18)
// at Object.runInThisContext (node:vm:309:38)
// at node:internal/process/execution:77:19
// at [eval]-wrapper:6:22
// at evalScript (node:internal/process/execution:76:60)
// at node:internal/main/eval_string:23:3
const name = 'Will Robinson';
console.warn(`Danger ${name}! Danger!`);
// Prints: Danger Will Robinson! Danger!, to stderr

Example using the Console class:

const out = getStreamSomehow();
const err = getStreamSomehow();
const myConsole = new console.Console(out, err);
myConsole.log('hello world');
// Prints: hello world, to out
myConsole.log('hello %s', 'world');
// Prints: hello world, to out
myConsole.error(new Error('Whoops, something bad happened'));
// Prints: [Error: Whoops, something bad happened], to err
const name = 'Will Robinson';
myConsole.warn(`Danger ${name}! Danger!`);
// Prints: Danger Will Robinson! Danger!, to err

@seesource

console
.
Console.log(message?: any, ...optionalParams: any[]): void

Prints to stdout with newline. Multiple arguments can be passed, with the first used as the primary message and all additional used as substitution values similar to printf(3) (the arguments are all passed to util.format()).

const count = 5;
console.log('count: %d', count);
// Prints: count: 5, to stdout
console.log('count:', count);
// Prints: count: 5, to stdout

See util.format() for more information.

@sincev0.1.100

log
(`Hii worker #${
worker: WorkerClient<boolean>
worker
.
WorkerClient<boolean>.workerId: number
workerId
} from ${
vars: {
workerId: number;
}
vars
.
workerId: number
workerId
}`),
{
workerId: number
workerId
:
const client: WorkerClient<boolean>
client
.
WorkerClient<boolean>.workerId: number
workerId
}
);