Skip to content

Instantly share code, notes, and snippets.

@blakejakopovic

blakejakopovic/nostr_query_engine.md

Last active March 26, 2023 12:14
Show Gist options
  • Save blakejakopovic/a5f46cb42eb7da2c22c8af620ac37bb7 to your computer and use it in GitHub Desktop.
Save blakejakopovic/a5f46cb42eb7da2c22c8af620ac37bb7 to your computer and use it in GitHub Desktop.
Download ZIP
Nostr missing event query/request features and pain points
Raw
nostr_query_engine.md

Pain Points / design limitations

  • Limit is useless for subscriptions with multiple filters (as how to prioritise which filter results are most important? Left to right?)
  • Hard to do pagination (using since/until, but what limit to use? What if older events shared after you fetch, you’ve now missed it)
  • Race condition between EOSE and CLOSE subscription when you don’t want to stream (option to only return existing data only, without newer broadcast results)
  • Events from new pubkeys are missing basic kind0 data (ideally embed name/display_name for first pubkey event)
  • Websocket congestion control and bottlenecks (single event streamed at a time, blocked by larger events FIFO)
  • Unable to query for ranges of kinds (e.g. {"kinds": [30000-39999]})
  • More?

Missing Features

  • Option to request metadata for events (like reaction or reply counts, etc)
  • Reverse sorting capability
  • Ability to query with POW minimum (today only starts with N zeros, and support for that looks to be EOL as a supported query)
  • Unable to request related events be included in the same request (e.g. include all replies to event with depth x)
  • Relay pre-filtering (clients should enforce filter checks locally, however this can reduce data usage)
    • Keyword filtering (we have NIP-50, however I think we can improve if we review a new request structure)
    • Tag filtering (e.g. without "content-warning")
    • Pubkey muting (eg. share cuckoo filter of pubkeys to skip/ignore)
  • Ability to share cuckoo filters of received events (maybe per relay session - eg. only send profile kind 0 once, no need to send again unless replaced)
  • More?

Questions

  • How much caching or deduping logic in server side vs client side - how do we balance and perhaps allow disable server logic?
  • Can we perhaps pre-can some common client app queries so relays can cache the data for better performance? Perhaps in areas like DMs, notifications, etc.
  • Do we need to consider relay indexers/aggregators and any needs they may have?
  • Potential for perhaps client config (e.g. a mobile client can request more data saving features be enabled)
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment