the Grapevine Roadmap

[PrettyGoodFreedomTech]

This thread will be used to update and discuss what we are currently focused on building -- the next baby step -- including how it fits into the overall roadmap for the grapevine.

[PrettyGoodFreedomTech]

It is important to distinguish between:

• where we are going and why it's important
• the roadmap from here to there
• milestones on the roadmap
• the next baby step

I'm going to define the goal as the ultimate realization of the cypherpunk vision of freedom through digital privacy, with our focus being on decentralized reputation and web of trust. I could talk about the tapestry protocol (or method or theory or whatever), but that is for another thread.

In this thread I am going to touch upon all of the above, but the focus will be the next baby step. It will probably not be uncommon that the next baby step turns out to be bigger than anticipated, in which case we break it down into smaller steps. Also, the big picture may evolve, inw hich case the baby step will need to evolve along with it.

[PrettyGoodFreedomTech]

Previous work

The grapevine algorithm has been implemented at brainSToRm.ninja. Users can log in, calculate Grapevine Web of Trust ("influence") scores, legacy WoT scores, and Degrees of Separation (DoS) scores, explore the results in table format, and use those scores to curate wiki content using the brainstorm app: nostrapedia. Unfortunately, the multiple steps and time required to generate the scores limits usage.

To improve the user experience of score calculations, @manimejia has built grapevine.my which triggers all of the necessary steps (download user follows, download their follows, calculate scores) in a single click. In addition, the single click also triggers exportation of the top 100 npubs (not including existing follows) as a NIP-51 list, which can be imported at various sites including coracle, amethyst, and listr. Usage has improved but remains limited.

Observations and lessons learned:

1. Today's legacy web of trust solutions throughout the nostr ecosystem rely on simple calculations that can be done on the fly at the front end. Brainstorm.ninja reveals that this is not feasible for the grapevine. It takes too long to download all of the relevant follows and mutes data, plus an extra few minutes to do all of the grapevine calculations.
2. NIP-51 lists are not big enough to store all of the scores as individual lists. Most if not all nostr relays will reject them unless we break them up into multiple smaller lists. How to break them up at the protocol level is uncertain.

Some proposed solutions:

1. Grapevine calculations need to be handled on the back end, ahead of time, before the scores are ever used to curate content.
2. Break one big list of npubs into multiple smaller NIP-51 lists, grouped together using a single master NIP-51 list. Downside is that most clients will not recognize whatever protocol we implement to do this.
3. Specialized Grapevine relays that can store large NIP-51 lists, which can be used directly by most nostr clients without the need for new protocols.

Current focus ("baby steps")

I am focused mostly on fleshing out the big picture roadmap with a focus on an upcoming major milestone: grapevine's modular structure, and have been working closely with @manimejia on what he is doing (below). Also, over the past 2 or 3 days I have been laying the groundwork to refactor brainstorm.ninja. At some point, we're going to begin to need the concept graph, and I'll need brainstorm to experiment with different concept graph implementations. Unfortunately, brainstorm.ninja is currently bloated with lots of technical debt:

• The primary technical debt is the result of caching too much nostr data in the browser, using redux, using several "listeners" which are slow to process events.
• Second, I've too much intermingled the processing and the display of data, which results in react component rerenders getting out of control.
• Third: I was using the nostr-react library but this has not been maintained and will be switching to nostr-hooks which is being actively maintained and improved. One thing I never figured out prior to using nostr-hooks: how to publish a note to a specific relay. This will be important once we start developing the concept of Grapevine Relays, both personal and GRaaS.

As a first step towards refactoring brainstorm.ninja, I am building the PGFT Nostr + React + Dashboard Template, which will rely on osty's nostr-hooks. If I can polish this template up well enough, I may promote it to the wider community -- the "PGFT NRD Template" -- as something that may be of use to other devs.

@jeremyd has refactored the front-end, javascript implementation of the grapevine algorithms at the brainstorm repo for use by relays at the back-end using functions written in go and stored in the gvengine repo. Goal: Progress towards construction of the world's first GRaaS: Grapevine Relay as a Service that will maintain kinds 3 (follows) and 10000 (mutes) events, along the lines of purplepag.es, and use them for score calculation. The rationale for merging these two functions is speed: a new user can sign up for GRaaS, request grapevine scores, and have them calculated quickly, since most if not all relevant follows and mutes will already be present in that relay.

@manimejia is focused on rebuilding and expanding the existing functionality of grapevine.my with several improvements:

• the calculations will be done on the back-end, not the front end
• choices of what exactly gets included in the exported NIP-51 lists (see below)
• free and paid versions

Goals:

• better user experience, wider usage, greater visibility of the grapevine project
• proof of monetization

Step 1 will be to produce a single NIP-51 list, just like at grapevine.my: the top 100 npubs, minus the npubs that are already in the user's follows.

Step 2 will provide a wider variety of NIP-51 lists. Options may include some or all of the following:

• include / exclude the user's existing follows
• grapevine WoT score vs legacy WoT score vs Degrees of Separation
• options to adjust GrapeRank parameters (attenuation factor, rigor)
• number of npubs (100, 500, 1000)
• which page to export (top N, bottom N, page M of N, etc)
• define an influence score cutoff (currently all scores > 0 are included)
• break the entire list down into multiple NIP-51 lists, with an additional master NIP-51 list to wrap them all together
• incorporate reports (kind 1984)

Undecided how much to flesh out step 2 before proceeding with next steps.

next steps

One of the first major milestones is going to be to separate the grapevine into distinct modules as in the figure below.

Why modularize? My intuition tells me this is going to be extremely important. Not sure I can yet articulate why.

At what point should we focus on the separation-of-modules milestone ? This remain to be seen. For now, we should focus on these two minor milestones first:

1. Refactor grapevine.my (@manimejia), which merges the Interpretation Engine and Calculation Engine together, and does not require specialized grapevine relays.
2. Build the world's first GRaaS (@jeremyd), which will effectively merge the functions of ALL of the above modules into one: vanilla relays, the Interpretation Engine, the Calculation Engine and the Grapevine Relay.

[wds4]

Updated Roadmap

(proprietary-ish?) Product 1: Google for Nostr; PageRank for Pubkeys

• alternative to nostr.band
• nostr keyword search, with results stratified using PageRank
• variation on the idea outlined above of GRaaS (Grapevine Relay as a Service) (may or may not technically be a relay)
• centralized nostr cache
• incorporates neo4j which will facilitate scaling
• calculate PageRank (using Brainstorm as the observer)
• calculate GrapeRank from the point of view of customers
• view DoS between any two pub keys
• single pubkey profile view with various scores:
  • numFollowers (includes bots) (observer not required)
  • numFollowers adjusted (numFollowers minus the bots; this is effectively GrapeRank "input") (observer is required)
  • DoS (observer is required)
  • PageRank score (observer is required)
  • GrapeRank score (observer is required)
• view Grapevine WoT in table format
• API to provide data in various formats

FOSS Product 2: Personalized GrapeRank WoT Relay

• alternative to utxo's WoT Relay
• two possible strategies (might want to implement both):
  • fork of fiatjaf's relay (which is what utxo did)
  • implement as a plugin for strfry
• will borrow many elements from Step 1, including use of neo4j

FOSS Product 3: Worldviews

• build tools so devs can easily crank out novel interpretations and incorporate them into novel Worldviews

Product 1 will be a centralized service that will make money the same way as legacy platforms. This will not be profitable unless we can expand the user base of nostr dramatically. We will be incentivized to build out FOSS Products 2 and 3 as part of a strategy to attract users from legacy platforms into nostr.