Skip to content

Understanding sharding

Sharding

In seyfert the sharding approach is to give the full benefit of scaling while keeping the same structure in your project.

Why sharding?

JavaScript runtimes are single-threaded, which means that without sharding, all processing would happen in a single thread. This creates a processing limit where all the load is evaluated together. Seyfert handles sharding internally in the Client instance to help distribute this load across multiple processes or threads (although it is still an imperfect technique).

Managing shards

The base of the Worker is to allow to execute code in parallel in different parts of the CPU, either in threads or different processes. In terms of discord, this means the ability to connect several shards by spreading the load on each Worker, for seyfert this is just changing the mode property in the WorkerManager to decide the execution mode between threads to spawn clients on processor threads or cluster to spawn clients on different processes of the runtime.

import { 
class WorkerManager
WorkerManager } from 'seyfert';


const 
const manager: WorkerManager
manager = new 
new WorkerManager(options: Omit<PickPartial<WorkerManagerOptions, "token" | "intents" | "info" | "handlePayload" | "handleWorkerMessage">, "resharding"> & {
    resharding?: PickPartial<NonNullable<WorkerManagerOptions["resharding"]>, "getInfo">;
}): WorkerManager
WorkerManager({
  
mode: "threads" | "clusters" | "custom"
mode: "threads",
  // ./src/client.ts for bun and deno (?
  
path: string
path: "./dist/client.js",
  // you can override a lot of options, like number of workers, shards per worker...
});


const manager: WorkerManager
manager.
WorkerManager.start(): Promise<void>
start();

Too simple? Seyfert takes care of all the logic so your project shouldn’t change much just by switching to a WorkerSharding.

Cache

Unlike traditional Discord libraries, Seyfert offers unified cache management across all shards. The cache can be centralized in the main process (the WorkerManager executor), ensuring consistent data access throughout your application.

To implement centralized caching, use the WorkerAdapter:

import { 
class WorkerClient<Ready extends boolean = boolean>
WorkerClient, 
class WorkerAdapter
WorkerAdapter } from 'seyfert';


const 
const client: WorkerClient<boolean>
client = new 
new WorkerClient<boolean>(options?: WorkerClientOptions): WorkerClient<boolean>
WorkerClient();


const client: WorkerClient<boolean>
client.
WorkerClient<boolean>.setServices(rest: ServicesOptions): void
setServices({
  
ServicesOptions.cache?: {
    adapter?: Adapter;
    disabledCache?: boolean | DisabledCache | ((cacheType: keyof DisabledCache) => boolean);
}
cache: {
    
adapter?: Adapter
adapter: new 
new WorkerAdapter(workerData: WorkerData): WorkerAdapter
WorkerAdapter(
const client: WorkerClient<boolean>
client.
WorkerClient<boolean>.workerData: WorkerData
workerData)
  }
});


await 
const client: WorkerClient<boolean>
client.
WorkerClient<boolean>.start(options?: Omit<DeepPartial<StartOptions>, "httpConnection" | "token" | "connection">): Promise<void>
start();

Talking to other workers

If for some reason (I did not find any for the example), you want a specific worker to execute an action that another one received, you can simply ask it.

const client: WorkerClient<boolean>
client.
WorkerClient<boolean>.tellWorker<void, {
    workerId: number;
}>(workerId: number, func: (_: WorkerClient<boolean>, vars: {
    workerId: number;
}) => void, vars: {
    workerId: number;
}): Promise<void>
tellWorker(
  1,
  (
worker: WorkerClient<boolean>
worker, 
vars: {
    workerId: number;
}
vars) => 
var console: Console
The console module provides a simple debugging console that is similar to the
JavaScript console mechanism provided by web browsers.


The module exports two specific components:


    

  • A Console class with methods such as console.log(), console.error() and console.warn() that can be used to write to any Node.js stream.
    • 

  • A global console instance configured to write to process.stdout and
process.stderr. The global console can be used without importing the node:console module.
    • 



Warning: The global console object's methods are neither consistently
synchronous like the browser APIs they resemble, nor are they consistently
asynchronous like all other Node.js streams. See the note on process I/O for
more information.


Example using the global console:


console.log('hello world');
// Prints: hello world, to stdout
console.log('hello %s', 'world');
// Prints: hello world, to stdout
console.error(new Error('Whoops, something bad happened'));
// Prints error message and stack trace to stderr:
//   Error: Whoops, something bad happened
//     at [eval]:5:15
//     at Script.runInThisContext (node:vm:132:18)
//     at Object.runInThisContext (node:vm:309:38)
//     at node:internal/process/execution:77:19
//     at [eval]-wrapper:6:22
//     at evalScript (node:internal/process/execution:76:60)
//     at node:internal/main/eval_string:23:3


const name = 'Will Robinson';
console.warn(`Danger ${name}! Danger!`);
// Prints: Danger Will Robinson! Danger!, to stderr


Example using the Console class:


const out = getStreamSomehow();
const err = getStreamSomehow();
const myConsole = new console.Console(out, err);


myConsole.log('hello world');
// Prints: hello world, to out
myConsole.log('hello %s', 'world');
// Prints: hello world, to out
myConsole.error(new Error('Whoops, something bad happened'));
// Prints: [Error: Whoops, something bad happened], to err


const name = 'Will Robinson';
myConsole.warn(`Danger ${name}! Danger!`);
// Prints: Danger Will Robinson! Danger!, to err
@see ― source
console.
Console.log(message?: any, ...optionalParams: any[]): void
Prints to stdout with newline. Multiple arguments can be passed, with the
first used as the primary message and all additional used as substitution
values similar to printf(3)
(the arguments are all passed to util.format()).


const count = 5;
console.log('count: %d', count);
// Prints: count: 5, to stdout
console.log('count:', count);
// Prints: count: 5, to stdout


See util.format() for more information.
@since ― v0.1.100
log(`Hii worker #${
worker: WorkerClient<boolean>
worker.
WorkerClient<boolean>.workerId: number
workerId} from ${
vars: {
    workerId: number;
}
vars.
workerId: number
workerId}`),
  { 
workerId: number
workerId: 
const client: WorkerClient<boolean>
client.
WorkerClient<boolean>.workerId: number
workerId }
);